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CHAPTER I 
INTRO/JUCTION 

The close procedure has In recent years been receiving considerable 
attention* This attention has primarily been focused upon two aspects of 
the use of close procedure , namely * readability and reading comprehension 
testing* The third aspect 9 that of using the close procedure as a tech** 
nlque for teaching reading per ee in various reading imps went situations* 
has also begun to be colored, The present study deals with this aspect of 
the close procedure as a teaching device, but by virtue of the dose pro- 
cedure itself, the other aspects will in some measure also be investigated. 

Definition of the , dose Procedure . To define the dose procedure 
in terms of its construction is a relatively simple matter* To determine 
what this particular structure does when applied to pupils is considerably 
more complex. The dose procedure is basically a systematically mutilated 
passage delivered to students with the instructions to fill in the blanks 

with the word which is missing from the text. The systematic deletion is 

/ 

generally accomplished by selecting one of the first words in the passage 
by a random procedure and then following this selection with the deletion 

of every Nth word. Commonly every fifth word is deleted from close pro- 
cedure material* 

For the present study, for purposes to be outlined below, every 
tenth word is deleted. In the original study of dose by Taylor it was 
shown that within these limits it made relatively little difference 
whether every fifth or every *enth word was ddeted for readability 
purposes. In addition to the systematic deletion, there is typically 
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a minimum deletion sample size required to give some reliability to tho 
measure* Particularly is this true when one is concerned with dose 
procedure as a readability or comprehension testing device* The minimum 
which seems to be commonly accepted is 50 deletions or 250 words, there 
is not an apparent maximum to the length of a dose procedure passage , 
but certainly such factors as fatigue and extinction suggest that this 
may well be an area to explore* 

There are essentially two ways to handle dose procedure materials, 
the first is ms a testing device; the second as a teaching device* The 
first involves no reinforcement, or knowledge of the correct response on 
the part of the responder* In essence, the individual fills in the do 20 
blanks and turns in his paper, and may or may not see it again. But if he 
does, it is simply in the shape of a mark or a score. 

The second is through a technique of reinforcement; here the 
individual achieves a knowledge of the correct response which he has made. 
This knowledge of correct response allows the individual then to accomplish* 

One must consider, however, that non-reinfcrced dose procedure is 
not completely without reinforcement* Certainly the individual who fills 
in a dose blank. can, by his own means by matching his language pattern 
thus produce*' by his dosure, reinforce himself. Hence non-reinforced 
dose procedure is in effect non«*overt reinforced dose procedure. It is 
the effect of this non-reinforced dose procedure which is the topic of 
this present study* 

In essence, the concern will be to attempt to measure the effects 
of a prolonged series of non-overtly-reinforced dose procedure exercises 
upon reading comprehension and upon the ability to make closures* 
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Readability * The initial use of the cloze procedure by Taylor 
was to determine the readability of passages. The number of correct 
responses to the mutilation in a particular passage gives an indication 
of the difficulty of the passage* If such a passage is delivered to a 
group of people, and there is a significant difference between the mean 
number of correct closures of that passage and another passage delivered 
to the same or similar people, we can assume that the words which were not 
mutilated in the more difficult passage did not give a sufficient amount 
of information to the reader to allow him to make the closures without 

M 

making errors* Essentially then, at one word in five deletions, we are 
saying that eighty percent of tho words in one passage contained less 
information regarding the total meaning of the passage than in another 
case* V/e may also infer that either redundancy is reduced or that tho 
language pattern used in the passage is unusual and does not allow the 
individual to guess the proper word using his available language ability* 
In either oase the passage is more difficult to read* 

Two important questions arise in relation to measuring readability 
by doze* First, does it make any differenee whether one scores synonyms 
as correct or whether one simply scores as correct the exact word deleted? 
Taylor 1 s early work shows there is essentially no difference in the rank 
order of passages, regardless of the scoring system used* 

The second question which arises is: Dees the deleted word make 

any differenee in the outcome of the readability estimates? Here Bormuth's 
study ^ which systematically deleted each one of the words in five differ* 
ent forms of a doze procedure selection indicates there is no difference 
as a function of which words are loft out if the deletions are done in the 
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systematic doze procedure fashion* Cloze procedure readability essentially 
is comparative* and what is needed in this particular area is some variety 
of standardization so that given a particular level of responding* x number 
of errors* or £ proportion of errors* will represent a particular grade 
level or age level of reading for a particular child* Essentially the 
relationship between the proportion of deletion errors in a doze procedure 
passage and th Ability of the child at a particular level in school to read 
the material needs to be more fully explored* 



Comprehe nsion Testing * Just as different passages prove to be of 
differential difficulty for groups of people* so too does the same passage 
appear to be of differential difficulty for different people* With this 
as a premise* the doze procedure can be used as a reading comprehension 
oesting device* Large numbers of studies have used doze procedure for 
testing reading comprehension. Early studies in this areas were by Taylor^ 

pQ 

and by Rankin. With one or two notable exceptions, these studies find a 
reasonable correlation between the number of correct responses on a doze 
procedure passage and reading comprehension test scores* The interesting 
thing about doze procedure as a comprehension testing device is that it 
allows the analysis of the intemd comprehension process as it is a tech- 
nique for asking continuous comprehension questions during the reading of 
the passage; hence* in relatively long passages* may be analyzed fo~ a 
differential in reading comprehension in various parts of the passage, or 
may be systematically altered to detezmine some of the spedfie variables 
which may enter into reading difficulty, both in relation to the nature of 
the material itself* and also in relation to the nature of changes in the 
individual reader as he goes through a particular passage. It is dear 
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that reading comprehension depends upon the ability of an individual to 
prediot words which have been deleted from a particular passage. Hence 
the aspect of reading comprehension measured is the concurrence of the 
language pattern of the reader with the language pattern of the writer, 

Where these two concur the reader is well able to estimate what words the 
writer will use and the amount of change which must take place in the 
reader in order to understand the particular passage is reduced* The 
oloze procedure seem*? to fit the concept of learning to comprehend by 
reading phrases and avoiding vord by word silent reading, or even oral 
reading, of the passages* Where closures are readily made, the language 
ability of the individual is developed to the level of the particular 
writing. The concepts tend therefore to be fairly well formed and cloze 
procedure reading comprehension is a measure, not so much of how much the 
individual oan learn from a passage but of how much he already knows about 
it when he enters the passage. The reverse is true when we consider the 
typical reading comprehension selection which consists of a paragraph 
followed by a series of questions about the paragraph which the individual 
must answer. Here we know relatively little about the process by which the 
individual comes up with a correct answer, and we are measuring not so much 
his reading comprehension ability directly but how much he is able to learn 
or retain from the particular passage. While these two aspects are quite 
obviously correlated and are even more highly correlated when a time limit 
is put or. the normal reading comprehension situation, they appear also to be 
somewhat different— the one dealing with existing concepts, the other deal- 
ing with the ability of the individual to develop concepts from reading 
itself. Perhaps what is needed here is a definition of reading comprehension 



o 

ERIC 



6 

as either one or the ©the? of these aspects, in essence, is reading 
comprehension to be measured by the number of extant concepts which an 
individual holds in common with the reading material, ©r is reading com- 
prehension the ability of an individual to learn new cone opts from the 
material? In either Instance, the reading process via the cloze procedure 
whore an individual must stop periodically and insert a word is a differ- 
ent procedure from the straight reading that we are accustomed to. 

Cloze Teaching , It is this difference between the cloze procedure 
and the reading process which makes possible the use of cloze procedure 
as a teaching device for improving reading comprehension. For teaching 
purposes reinforced doze procedure is essentially a continuous stream of 
short-answer questions, the answers to which must be derived from the 
reading material which surrounds the blanks with terminal reinforcement* 

The individual, therefore, who does a cloze procedure passage is required 
to focus somewhat more upon the matt rial than the individual who is simply 
reading the material without regard for blanks. Therefore, if the concepts 
are already available in the learner, doze procedure forces him to derive 
actively the concept from his own knowledge, whereas continuous reading 
allows the individual to absorb the ooncepts passively from the writer* 

The effects of the focusing required in cloze procedure are somewhat debat- 
able* If an individual does not have the available language patterns and 
sufficient cues are not presented in the reading material, the individual 
may have difficulty understanding a doze procedure passage; hence, there 
will be loss in comprehension* If the individual can manage to fill in 
most of the blanks in the doze procedure material, the focusing on the 
material conceivably may have the effect of making the reader more attentive 
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to the material he in reading. Such a procedure has the effect of 
reducing the rate at which individuals read but the overall time saved 
by good comprehension may more than make up for this time loss. On this 
basis several experiments have been performed which tend to lend some 
credence to this particular position.* Each of these studies 

involved the use of feedback to the pupils in terras of the correct answer 
which was delivered immediately after the pupil had completed the doze 
exercise. The feedback of the correct response would give the individual 
reinforcement or corroboration for the responses he made and allow him to 
determine where his particular errors were made. For present purposes it 
is necessary to examine the doze procedure somewhat more dosely. If 
the doze procedure is a series of short answer questions about a bit of 
material which is at once both tho questions and the souroe for the answers, 
then, excepting format, the doze procedure is very similar to programed 
instruction, the difference being that a large number of answers are made 
before the feedback occurs (rather than Immediate feedback, as is the case 
with programed instruction). The Roosniok study^ made use of this 
immediate feedback for each of the items and found that a very low error 
rate oould be devdoped for materials designed for fifth and sixth grade 
pupils. 

Let us for a moment examine the meanings of the word closure . 

Closure is a Gestalt term which suggests that when a situation is left 
unfinished the individual himself tends to finish it— -in essence, to dose 
the gaps, to make the material •good*. Essentially this is an internal 
process and is not as such available to sdentifle scrutiny. However, if 
one conjectures on the meaning in psychological terms of "making things 




good"* this goodness &ay be construed as a state of self-reinfor octaent . 
Closure and the subjectively described positive affect state which comes 
idth getting an answer which to the responder fits the particular situa- 
tion nay well be a situation in which the individual can reinforce him- 
self, In essence, if the language pattern which he develops fits both 
bis known language pattern and makes sense to him in terms of what the 
writer has put down, be automatically gets an amount of feedback irres- 
pective of whether objective feedback is available. If the preposition 
that closure represents self-reinforcement is in fact tenable, then any 
doze procedure material would in turn be a teaching device, albeit 
perhaps not as effective as the same material where overt feedback is 
delivered. 




gloss Procedure and linguistics. While Bormuth's study has 
shown that the systematic random deletion of words does not affect the 
scores one would obtain upon close procedure materials, an analysis by 
Bloomer and Heitcman of the errors mad^, by syntax of deletion, indicated 
a differential in the probability of error for the various syntactical 
elements. In effect this is saying that the deletion of words of a 
particular syntax over another makes a close passage more or less dif- 
ficult* If we look at the deletions in terms of the contribution to 
the meaning of the passage we find that words of some particular syn- 
taxes (in this case nouns and adjectives) tend to contribute more mean- 
ing to the passage than do other word forms, particularly the function 
words. The Bloomer and Heitzman study employed random deletions. By 
controlling the relative difficulty of the passage by the syntax of 
deletions, the ability of the individual to learn from the material 
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and to match language patterns with the writer can be measured in relation 
to the contribution to meaning of words of valuing syntaxes. In addition 
the nature of the substituted words (the error words) will indicate some- 
thing of the nature of the language pattern of the individual. 

fhe present study proposes to investigate an aspect of each of 
these four areas— close readability , close comprehension* dose teaching, 
and close linguistics— by presenting twenty-four different selections, 
followed by comprehension questions* in one of the following dose forms: 

1. Control (no deletions) 

Zm Random ddetions 

3. Noun ddetions 

4. Verb deletions 

5« Adjective deletions 

6. Preposition and conjunction deletions 

7. Artide deletions 



CHAPTER II 



LITERATURE 



III© experimental literature dealing with the cloze procedure now 
approaches fifty articles and dissertations* An excellent review of 
these materials is given in Rankin's survey in the 1964 National Reading 
Conference Yearbook* The present review of the literature will restrict 
itself to studios pertinent to the present investigation* Tho treatment* 
by Rankin is somewhat more exhaustive in the types of studies covered. 

Readability* Table I gives the studies of cloze procedure as a 
technique for measuring readability* In reading the table 9 under the 
heading "Cloze Method" 9 any word refers to the random deletion of words 
regardless of their syntax* Unless otherwise specified in this column, 
the oloze method is given either first or independently of the variable 
which is measured against it. Noun-verb deletions represent passages in 
which only nouns and verbs have been deleted* The evidence in Table I 
indicates that the doze procedure can be used to measure re adabi lity* 
Cloze has been used with some success in several languages as well as 
English and when a passage is given several times a reliable estimate of 
reading difficulty which can be used for rank ordering is available* 
Essentially the work which has not yet been done on the cloze procedure 
as a readability technique is to establish the timber of correct doze 
responses to be anticipated at a particular grade level* In essence, 
standardization of the doze readability technique is not available* 
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General Reading Achievement, The do 550 procedure has been 

extensively examined at a technique for testing reading achievement. 

Table IX gives a summary of the results of the various studies* The 

dose procedure gives respectable correlations with reading achievement 

46 

in each case save the Weaver and Kingston study and of course dose has 
been used as a reading comprehension testing device in the Stanford 
Achievement Test* 

Hi the present study the dose sdections are compared against 
standardised reading tests given both pre-test and post-tent* The fact 
that syntactical deletions of a single type are found in particular 
selections allows the determination of the relationship between reading 
oompraheneion scores and particular syntactical deletion, in addition 
to the anywtiord deletions which have been oustomarily used. 

Close and Comprehension of the Same Materials* Table III summar- 
ises a series of studies which relate the number of correct dosures with 
comprehension tests covering the same materials* The relationships are 
relatively high except for the study by Friedman^ 5 which shows that when 
individual scores are related to comprehension test scores the correlations 
appear to be quite low* One would expect this a 0 the scores of an indi- 
vidual are less stable than those of a group and less likely to represent 
a true score* At the same time it indicates that the reliability of doze 
for individual testing may be rdatively low* 

Each of the dose sdections in the present study also has a 
comprehension test given immediatdy after It* As a consequence, 
relationships, both individual and group, can be determined for dose 
material with deletions of varying syntaxes* In addition the relationship 
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between the various doze forms and straight unmutil ted reading of the 
same material in relation to the comprehension is determined* 



doge Procedure and Personality * A study by fiankln has shown 



that cloze procedure scores are somewhat lass reliable for introverts 
than for extroverts* In the present study the California Test of 
Personality is given to determine the interaction between personality 



variables and cloze deletions as well as reading comprehension with 
various personality characteristics* 

doze Procedure* Verbal Ability and IQ * Table IV indicates that 
the doze procedure is strongly related to the number of verbal measures 
as well as the number of intelligence tests or sub-tests. The measure- 
ment of IQ is included in the present study primarily as a support for 
these materials* 



doze Procedure as a Teaching Device , Several studies have been 
done using the doze procedure as intermediary between pre-test and 
post-test, the purpose of the doze procedure being primarily as an 
intervening variable* Bach of the four studies reported indicates in 
one way or another that doze procedure may be usefd as a teaching 
device* In one instance there was no significant difference between the 
gain of a straight reading group and the gain in a doze reading group; 
in the second instance significant differences were found in favor of 
people taking doze procedure material as opposed to controls with no 
reading experience* In each of the studies reported, some kind of 
motivational device was used* It is very difficult to separate the 
motivation and the possible benefits accruing therefrom from the effect 
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of the close material itself* If we look upon oloze procedure as self- 
reinforcing we wight anticipate that where motivation is reduced to a 
minimum, the motivation that may come from making correct close responses 
may produce similar effects* In the present study all attempts were made 
to reduce the motivation through the manipulation of the doze procedure 
in one form or another* Aside from the mere passing out of papers in 
the classroom, the children were in no other way stimulated to do the 
doze procedure in the present experiment. In addition, since they 
received no external feedback in relation to responses, motivation from 
rewards of being told they did well or poorly was eliminated from the 
experiment. This procedure allows us to determine whether the effects 
found in previous studies were a function of the combination of doze 
procedure and motivation, or whether the results were more related to 
the doze materials themselves* In addition, by using deletions of 
various syntaxes which in turn control the level of difficulty (Bloomer 



and Heitzman) we are better able to determine the effects of varying 
difficulties as well as the varying syntaxes in the present study. 

SUMMARY 

In general, the literature shows strong support for the use of 
doze procedure as a readability tool* It is also dear from the studies 
presented that the need is strong for the standardization of doze pro- 
cedure in terms of the number of errors per grade level so that a doze 
score will establish a grade level* Cloze procedure scores are related 
to reading comprehension scores under a variety of conditions. Cloze 



procedure has been found to be strongly related to the verbal aspects of 
intelligence* This would be expected from other sources of evidence* And 
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finally, doze procedure has boon used ad a successful reading compre- 
hension teaching device by several investigators* However, it is 
difficult to establish the differential offsets which may be attribut- 
able to motivation for other than the doze procedure materials them- 
selves* 



On the basis of this literature the present study presents data 
on a number of passages of relatively equivalent established readability 
for pupils of the fifth, seventh, ninth and eleventh grades* In addition 
the study presents data relating to deletions of varying syntaxes and 
the reading comprehension of both standardized reading tests and reading 
comprehension tests built for specific doze selections* 

Third, the study examines an area which is only touched tangen- 
tially in the literature, that is to say, the relationship of age and 




sex to doze soores* 

Fourth, the study examines' the relationship of intelligence test 
scores to doze procedure scores, as well as reading comprehension scores 
on the same material* 

Fifth, the study examines the effect of presenting a prolonged 
sequence of doze procedure materials under control conditions where 
motivational arrangements of material as well as overt feedback are not 
presented to the pupils, and where the difficulty of responding is 
controlled by controlling the syntax of the deletions* 
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CHAPTER III 



PROCEDURE 

GENERAL PUN 

The questions to be asked fell into two general categories— 
those which required a cross-section of several grade levels for 
relatively short exposure to close procedure materials * and a second 
series of questions which required longer time exposure to close 
procedure materials in a more longitudinal fashion* 

The. Cross-Seotional Experiment * Hie cross-sectional experiment 
involved pupils from the fifth* seventh* ninth and eleventh grades* 

The number of pupils Involved by grade level for each sex is given in 
Table VI* The pupils in the cross-sectional experiment were involved 
with the experimental material for three weeks* Six dose procedure 
selections were administered to the pupils* two each week on non- 
consecutive days* The data for the cross-sectional experiment involved 
the number of correct answers to comprehension questions asked by the 
various forms of the dose procedure materials* 

The Longitudinal Experiment * The longitudinal experiment was 
conducted with ninth grade pupils* The pupils were pre-tested with the 
differential aptitude battery and the California Test of Mental Maturity 
For twelve weeks they received two dose procedure selections delivered 
on non-eonseeutive days for a total of twenty-four* This was followed 
by dose procedure testing, the language and reading sections of the 
Iowa Test of Basic Skills* end an Information retention test* 
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Hare student absence became an Important variable in the experiment 
and any student who did not oomplete eighteen of the twency-four dose 
procedure tests or who was absent for any of the pre-tests or post- 
tests was eliminated from the experiment* 

Population and Sample, The pupils for the two experiments were 
drawn from a suburban central school in upstate New York, adjacent to a 
city of 350,000 inhabitants* The population was essentially middle class 
with a small proportion of rural pupils from nearby farms* The experi- 
ment involved a total of forty-nine classrooms in the eross-seotional 
and longitudinal experiments combined* The experiment involved the 
total population of classrooms in the school system at the fifth, seventh 
ninth and eleventh grades* 

TEST MATERIALS 

Pre-Tests. The pre-tests administered to the ninth grade pupils 
involved in the longitudinal experiment were the Differential Aptitude 
Test and the California Test of Mental Maturity* The Differential 
Aptitude Test was chosen because it gives a reading score in addition 
to numeric*! and spatial relation scores which were to be correlated 
with the dose procedure data* 

The Post-Test* Tho poet-test consisted of one standardised test 
and two tests developed specifically for the experiment* The standard- 
ised test was the Iowa Tent of Basic Skills (the language and reading 
sections),, the language section giving correlational data to relate to 
the dose procedure scores in the various syntactical deletions* The 
reading test scores showed differences in reading ability between groups* 
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The Final Comprehension Teat, The final comprehension test 
consisted of 144 multiple ohoioe questions In groups of six* which were 
taken directly from the compehension questions presented following each 
of the doze selections* Since the amount of time since a pupil had 
read the selection varied in systematic degrees these questions afforded 
a measure of recall over a period of twelve weeks* This test was admin- 
istered the thirteenth week* The questions were selected at random from 
the twelve comprehension questions following each article* Control was 
obtained for differences between materials and differences in questions 
as a function of the mixed orders in which children took each cloze 
procedure selection* 

The Cloze Procedure Final* The doze procedure final was a doze 
/selection, one thousand words in length* which was designed to determine 
whether or not there were differences in the ability of individuals to 
fill in doze procedure blanks as a function of the training which had 
occurred from the twenty-four preceding articles* This cloze procedure 
selection was essentially a random deletion selection which was produced 
in the same fashion as the oloze procedure material used in the experiment 
itself* The exception to this was that the number of deletions was bal- 
anced according to syntactic type* There were twenty deletions of each 
oxperimental syntactic type in the test* This test was graded according 
to the number of correct responses per type of deletion* and hence differ- 
ences between the ability of pupils with different deletion type 
0 

experiences could be determined* 
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Limitations of the Testing Program . The pre-test administered 
to the subjects were those tests customarily delivered by the school 
system* It was felt that additional pretesting would add an uncfoe burden 
on teachers and pupils* The post-tests * however* were dictated by the 
needs of the experiment. 



Selection. Twenty-four selections of six to seven hundred words 
in length were used in the experiment. Each selection contained a com- 
plete episode in that they were judged to. be: 1 ) understandable without 
previous knowledge of the area or characters or the subject; and 2) 
conflicts and issues raised in the material were resolved within the six 
to seven hundred words* 



Readability* The readability level of each of the articles or 
selections used for the experiment fell within the range of fifth to 
sixth grade on the Yoakum Readability Formula* On several occasions this 
required an adjustment of the readability of the article* Typically the 
readability was adjusted downward* The adjustment was made by substitut- 
ing words under the following conditions: 

1) proper names were changed; 

2) The substituted word must be a synonym of the deleted word; 

3) The substituted word was both shorter end more famili ar than 
the original word; 

4) If a word occurred in several places and this word was changed* 
the same substitution was made throughout the selection; 

5) There was no change in the denotative meaning of the passage 
as the result of the change in the words; 

6) The oonnotative meaning was as close to the original connotative 
meaning as possible; 

7) Ho more than six different words were changed in any single 
selection* 

These procedures allowed the construction of twenty-four separate pass- 
ages of equivalent readability without a change in the meaning or 
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dl struct! on of the author g s intent. Such a procedure was essential since 
the experiment was designed to compare each of the twenty-four exercises 
against the other exercises as they were presented sequentially to the 
children. Since the sequences of presentation to the students were 

essentially random,, a deviation of readability would make the scores not 
comparable* 

Comprehension Questions. Since it was the intent of the experiment 
not only to examine the dose procedure jjg£ se but also to jxamine its 
effect upon the reading comprehension of material, twelve cm, prehension 
questions were developed for each of the selections* Six of these 
questions were factual and six were inferential* The questions were 
multiple choice with four options* The correct response position was 
assigned at random* The factual questions were questions about material 
contained directly in the selection* The inferential questions required 

that the respondent infer from the material in the selection a most 
probable answer* 

Pre-Testing the questions. The quality of the questions and the 

% 

ability of the questions to discriminate was Important for the study* 

Each selection and its comprehension questions were pretested with seventh* 
grade students under the following conditions: 

The same class of students was used for pretesting all the 
materials* The pupils were a single class of seventh grade 
pupils in the junior high school of a snail Connecticut city 
with a population of about 20,000. The class selected was of 
average ability, being the middle seventh grade section of an 
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ability-grouped class* lbs materials wars presented to the 
students during a reading class* two selections were presented 
each week on non-consecudve days for twelve weeks* The pupils 
were instructed to read the selection as carefully as possible 
and to answer the questions which followed them* The data was 
collected on mark-sense cards processed through the test 
scoring program on a 1620 computer* The 1620 computer provided 
scores for each of the students* and an item analysis which was 
used to alter the comprehension questions and to control for 
difficulty with the following restrictions: 

1 ) Each option or distraotor must draw at least one response; 

2) The number of oorrect responses must be between sixty and 
seventy percent* to account for the faot that some of the 
materials are intended for fifth* ninth and eleventh grade 
pupil 8 as well as seventh grade pupils; 

3) The items must discriminate between better and poorer 
students significantly 

All items not meeting the above criteria were rewritten and the selections 
retested with another group of seventh-grade pupils with similar charac- 
teristics to the first* from a suburban township in central Connecticut* 

Deletions* Once the materials had been pre-tested and the questions 

met the above criteria* each of the twenty-four selections was prepared in 

six dose forms and one reading form* The dose forms were prepared by 

deleting every tenth word* starting with a randomly-assigned first word* 

It is typical to use a twenty percent deletion in dose materials; however* 

a ten percent ddetlon was used for the following reasons: 

1 ) Experiments by Taylor and others show that ddetions of ten 
or twenty percent make relatively little difference in the 
eoores of students; 
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2) If the twenty percent level was chosen, it would be virtually 
impossible to get equal numbers of deletions in the various 
grammatical categories* The deletions were made according to 
the following procedure for each of the forms: 

Form 1 * . Reading Selection: No words deleted* 

Form 2* Random Deletions : Every tenth word* starting with a 

randomly selected first word* was deleted regardless of syntax* 

Form 3. Noun De letions : Tan percent of the passage was deleted. 
Only nouns were deleted* Proper names and pronouns wo*%e not in- 
cluded in the deletions* The initial word was chosen by random 
selection* If the random selection was not a noun the nearest 
noun to it was ohosen* The noun nearest the tenth word following 
was likewise deleted* If two nouns were equidistant from the 
tenth word which was not a noun* the noun which occurred last 
was deleted* 

Form 4, Verb Deletions : Ten percent of the passage was deleted 
as with the nouns* The first word was ohosen by random selection* 
If this was not a verb* the verb nearest to the word was deleted* 
The rule for succeeding deletions was as above* Gerunds and 
auxiliary verbs were excluded from the deletions* 

£g£m.-5* Modifiers t Ten percent of the passage was deleted 
in the manner described above* The ten percent Included 
adjectives and adverbs* 

£grm_6«_ Function Wor ds— Prepositions and Conjunctions : Ten 
percent of the passage was deleted in the manner described above* 
The deletions were all prepositions and conjunctions^, 

7« „ Function Words — Noun Determiners ; Ten percent of the 
passage was deleted in the manner described above* The dele- 
tions were all noun determiners* 

Other Grammatical Cat egories : Two other grammatical categories 
were attempted— auxiliary and functional verbs* and pronoun sub- 
stantives* In neither case did they occur in sufficient number 
to allow ten percent deletion in all twenty-four selections* 



GRAMMAR 

A number of problems present themselves to any investigator 
who applies the cloze technique to some sort of grammatical schema* 

As it has heretofore been used* the oloze technique involves deletions 
of single words and depends for its effectiveness as a measure on the 
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degree of difficulty of replacing a single word, a variation which may 
be regarded ae poeing the problem of filling a smaller or larger closure 
in prose. To delete more than one word without designating the gap as 
a oomplex problem in whi ch more than one word is missing makes the task 
enormously more difficult for the reader who can in many cases fill a 
givan blank with a whole nominative or verbal construction® Numbering 
two or more oonsecutive blanks to indicate gaps of more than one word 
reduoes the difficulty somewhat but the reader is still likely to be 
misled. One of the chief problems, then, ie to build a design which takes 
as its basic unit of measure the word in a language not alwaye measurable 
in such units. For example. New York and put up with stand for a particular 
place and a particular act, and to ask which word is the noun or which word 
the verb in these multiple-word unite is to ignore the way they function 
in the language. Among function words, oi.e observes that three words, 
in apitaof. can be replaced with despite with negligible loss of meaning, 
but deletion of despite is likely to frustrate the reader unnecessarily 
oven when he knows what connective should be substituted— the three words 
he uses moire commonly to express this connection, in spite of. will not 
fit a one-word blank. The investigators found, r^tticularly among readers 
in the lower two grade levels examined, that there was an occasional 
tendency to fill in blanks with multiple word constructions that made 
grammatical senes and in some oases shewed a considerable sophistication 
in reading and digesting the general information conveyed in the context 
of the prose. 

3 y the same token, contractions such as that* a and there* s and 
1*22, be designated by a single blank, do In fact require • 
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a complex judgement by the reader, and, like despite , provoke a frus- 
tration in the reader who more regularly sees an aotor and an action 
represented in two words. A third problem is posed by separable verbs, 
which may occur far enough apart from each other to avoid the difficulty 
of consecutive word deletion, but onoe again are a single expression 
denoted as two word units. 

The traditional grammatical scheme of parsing sentences has the 
faults of any system which uses the interrelationships of one language 
to explain those of another vergr different tongue. Though some of the 
classifications of the older system are sound many more, particularly 
the parts of speech, are both unwieldy and misleading. Pronouns, for 
instance, are of various kinds and functions, and a description such as 
n a word that takes the place of a word designating a particular person, 
place or thing 11 , must be stretched considerably (even when one understands 
all the grammatical and lexical meanings in an utterance) if we are to 
account for demonstrative, genitive, interrogative and other uses of 
words so designated* 

It was not, however, the purpose of the investigators to account 
for all of the language, but rather to isolate every class of words 
which constitutod at least ten percent of the words in running prose so 
as to examine their individual contribution to the moaning of reading 
passages and to tost the effeots of thoir loss on grammatical and total 
meaning. This meant that seme kinds of words woro emitted from consid- 
eration, and because of difficulties already discussed some examples 
within classes wore regarded as unusable in the design of the experiment* 
Among nouns end verbs the traditional definitions wore used though the 
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function of individual examples was arrived at in a slightly different 
manner than the traditional methods of parsing. Ihoro was then a 
regular attempt to maintain consistency within a given class of words. 
j &g&Jte g and radar operator are not parallel constructions, though the 
first word in eaoh pair is commonly called an adjective, andi even such a 
recent study as that of Ruddoll^ makes the older description the basis 
for regarding jadar as a modifier in such constructions, the methods 
of identification of classes is based upon W. Nelson Francis' The 
§ taS9tBTe...0f American Bngliafr and Charles Carpenter Fries' Structure of 
ft 1 WTT-W ., Sr a t l ¥ n ?r . Ihese then were the classes of words deleted. 

gSUQft* Iho traditional definition of a word that indicates some 
sort of object, place name, or person was the guideline. In oases where 
the mechanioal system of mutilation pointed to a word which was Itself 
a noun but which was one functioning unit of a multiple word nominative, 
0.6. . SlS SW-yn Mftq . the word was skipped and the olosest single-unit 
nominative was deleted instead. Nouns were identified by a number of 
grammatical cues. Noun markers such as 4, thg, some, any, few, his, 
regularly signal a following noun even when other words intervene between 
the marker and the noun. Where morphology indicated that a given word 
might fall in one of two or more classes, syntax was taken as the gram- 
matical due. thus, "the lagfc man, " "a friendship which last£, » and 
"this homo was his laat" are differentiated by one or more of + h*"te 
methods. Both proper and common nouns were deleted. 

2g^s. ihia class was restricted to specific verbs, that is, 
function verbs such as <Jo and have in constructions such as "do drop us 
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a oard," and "havo you met ill as Jones?" wore excluded* 

An attonpt to construct dloae teats from which function verbs and 
auxiliary verba wore doloted proved Impossible for a large number of 
extended reading passages of various types* The number of words of 
this class fell far short of the percent neoessary for the deletlons 9 
and attempts to write or select passages with the necessary number of 
such verbs were rejected because readability was high* the Interest 
level or appeal of the passages for a variety of students was low, and 
the passages were predominately written In the historical past, proolud- 
ing the variety the investigators felt necessary for a general appraisal 
of the reading process* 
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SBSfllflc._modlflara> Ibis group Included both adjectives and 
adverbs, precluded the relatively small and fixed body of words represented 
in the above examples of noun markers, and verb markers* This group was 
differentiated by moans of the inflected endings, e*g*, ?ly. -er- -cat. 
and the syntactic relationship to words of classes one and two* 



gfcflPQgitiong and oonlimetl^ This body of words included all 
prepositions and conjunctions (except so-called " subordinating u conjunc- 
tions, words which are by themselves relatively empty of lexical meaning 
but do in fact point to relationships among other lexical units* 



This olasa comprised the articles a and the 
and also included certain possessive, demonstrative and interrogative 
pronoun uses (some of which are noted above as "noun markers") in such 
pronoun uses with nouns, e*g*, "that man," "whose glove?", %hat gall!" 
and "his mind, " as well, as some, anjr and Jew when they exercised a 
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determining affect cvoi? the nouns following them. The group is, in 
faot 9 another body of tl struoturd H words which do point out structural 
meanings and interrelationships among sentence constituents but exist 
because of their grammatical meaning rather than a lexical meaning. 

gronoun Substantives. The last class of words attempted included 
the substantive uses of pronouns when syntax indicated that they did not 
fall into the class designated "noun determiners" and where the individual 
words stood as complete markers for a nominal unit already evident* whether 
that nominal unit was one word or a noun clause or the whole thought 
denoted in one or more of the previous paragraphs. It also included 
expletive uses such as there in "there was a crooked man. • • * The follow* 
ing four examples of contrastive utterances will serve to illustrate the 
general method by which words of this class were differentiated* even 
though the same words were also categorised as noun determiners and they 
all occur in what are generally called the "subjects" of the sentences. 

Noun determiners: Some day my prince will oome. 

Many great hearts are asleep in the deep. 

Few men are called. 

Suoh men are true patriots. 

Substantives: Some Is better than none. 

Many are called. 

Few are chosen. 

Suoh. were the joys. 

Pronoun substantives did not oeour in the sample to a sufficient degree 
to warrant inclusion. 

Perhaps for the more sophisticated student of reading and writing* 
the most Important oategor y of words omitted from these classes Is the 
body of logical connectives represented in such words as 
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moreover, however and therefore . A teat of these function words which 
link a logical succession of utteranoes would probably increase our 
understanding of whatever mental shorthand the reader uses in compre- 
hending prose if we could construct a method for deleting a sufficient 
number of them to contrast with words of other classes. But again the 
investigators were confronted by an important class of words which 
constitute a very small percentage of the total number of words in 

running prose and tend to appear most frequently in rather difficult 
non-ficoion. 

While this scheme of classification is by no means an attempt to 
create an adequate grammar of American Ehglish it does describe the 
functions of more than ninety peroent of the words in the running prose 
examined. It will not begin to describe much of the structure of many 
modem colloquial works or non-standard dialects. It does provide a 
convenient and systematic method for deleting words of various kinds. 

ORGANIZATION OP MATERIAL FOR PRESENTATION 

To control for order of presentation effect and possible differ- 
ences in the materials! the twenty-four selections were randomly assigned 
to four blocks of six selections each, labeled A # B t C and D* Selections 
were presented to the students in blocks of six. For tho longitudinal 
experiment the first presentation thus included all four blocks of material 
These blocks were reassigned to different students on the second, third 
and fourth presentation of the longitudinal study. 

The longitudinal study was divided into four presentations of 
three weeks in length. The oross-seotional experiment was three weeks 
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in length* The u elections within each block were randomly assigned in 
sequence of presentation as a further control for order effects. The 
assignments within block sequence of selection was identical for all 
forms of deletion. The tables below show the analysis of variance 
between the blocks of material indicating no signify cant difference 
between the blocks and thus allowing grouping of the data between blocks 


for final computations* 












Table VII 








MEAN CORRECT COMPREHENSION QDESTXONS BX BLOCK 

AND GRADE 




Grade 




— _ Block 






Lev?!,,,.,,. , 


A 


B 


C 
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5 


5.64 


5*20 


5.04 


6.73 
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5.05 


6.46 


6.39 


7.90 


9 


6.90 


8.40 


6.60 


0.70 


11 


7.96 


8.64 


0.75 


6.90 
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ANALYSIS OF VARIANCE OF BLOCKS BY GRADES 
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Total 


23.76 


15 






^•95(3* 9 d.f.) 


* 3.06 








^.99(3, 9 d.f.) 


"6.99 
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The oross-sectional experiment was three weeks In length, and 
each student was assigned a single block of material. She students were 
assigned to blocks of material at random* 



| Assignment of Pupils* Each pupil in the experimental classes 

was assigned a block order and a deletion form by random procedures. 

\ Thus the experiment was replicated within each classroom. Pupils were 

| randomly assigned to treatments and orders. This procedure had the 

additional benefit of making it Impossible for one student to use the 
responses of another. Each block of material was inserted, in an envelope 
1 labeled with both its block and its form number* The envelope contained 

the following materials s 

1 • Six selections, each with the same deletion fom, and coded 
with a presentation number; 

2. Six sets of twelve questions, one set attached to each of 
the appropriate selections; 

3* Six IBM mark-sense cards coded to match the selections with 
the presentation order number. The mark-sense cards were 
pre-gKnohed with the selection order number, deletion type, 
and ctudent number; 

4. One IEM mark-sense name card to provide student name, age, 
grade and sex, coded as above; 

5* Six answer sheets with spaces for 7 5 answers, as well as the 
name, age, sex, grade, student number, and selection number. 



/ 

\ 



To limit outside interference and the possible Hawthorne effect 
of having observers in the reom v as well as to provide for generalisa- 
tion to other olassroom situations and to randomise the experimenter 
variable, the study was supervised by 59 classroom teachers* The 
envelopes containing the above materials, with the students 1 names 
written on them, were delivered to the teachers. 
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The teachers were instructed to; 

1 ) present the material to the assigned pupil at the beginning 
or class, two periods a week, on non-consecutive days; 

2) specify the material for the day by number, and be sure that 
the pupils used the oorrect IBM card; 

3) be sure that the students put their names anct other data on 
the answer sheet 

4) allow ar much time as neoessary to complete the material; 

5) insure that the students answering the comprehension questions 
did not refer back to the selection or to their answer sheets; ‘ 

6) provide seat work for the pupils when they finisVad their 
material; 

7) have the materials replaced in their envelopes anct returned 
to the teacher at the end of the hour* 

ffttpllg 1 Instructional The pupils 1 instructions were rend by the 
teacher and were as follows; 

r exercises will help you, in your understanding 

0 j tv “^Sllsh language* Prom your envelope remove Test ITo* 
and the packet of answer cards. 

"Now, find the brown name card and the answer card with the 
same number as the test on it. Pill in the name card with jour 
name, age, sex, and grade* 

"Now, remove one answer sheet from the envelope, fill in the 
date your name, age, sex, grade, test number, and the 

^Be sure your answer sheet and the I EM card have th© same 
number as th© exercises* Are there any questions thus far? 

- "f 11 ©zeroises you students have are different* Some 

of them will have no blanks* In this case read the exercise 
carefully and proceed to answer the comprehension questions. 

* ro rea <* the exercise. Wherever you find a blank. 

Sh01 J ld g0 ** the blank on yow answer sheet. 

. write the woru opposite the number on the answer sheet 

e 0r f eS S. 0 ?? f t0 the numbor of> tb© blank on the exeroise. 

Work as carefully as you can* You will have enough time to 
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finish. When you have finished the exorcise, turn to the compre- 
hension questions which follow* Answer these by making an appro- 
priate mark on your IBM card with the special pencil* Do not look 
back at the exercise but answer as well as you can from your memory* 
When you finish, raise your hand* I will collect your materials* 
Are there any questions? H 



SUMMARY 

Thus the treatments and orders of selection were randomly assigned 



to students, who filled in the missing words in the exercises on separate 



answer sheets and answered the comprehension questions during a regular 
class period under the supervision of a classroom teacher* 

TREATMENT OP THE DATA 

A total of 1411 students participated in the two experiments. 

In the longitudinal experiment a number of pupils did not complete all 
twenty-four exercises due to lack of ability, boredom, or absence* The 
students who did not complete at least eighteen of the exercises were 
dropped from the data analysis with the reasoning that any influence from 
sustained cloze procedure could not be detected with fewer exposures* 
There were 9,1 9$ completed sets of data on both the longitudinal and 
cross-sectional experiments* These included both the answer sheet and 
the mark-sense IBM card* Two whole classes were dropped from the experi- 
ment because the instructor felt the material too* difficult. Both of 
these classes were low-ability groups at seventh grade level. The total 
N for the longitudinal experiment was thus 114. The N for the cross- 

«N» 

sectional experiment was 976. .* , , . • 
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The data cards for each student were cleaned of stray marks 

and the marks made by the students were darkened prior to processing 

in the IBM 519 to punch the mark-sense data into the data cants* 

These cards were sorted by student naraber and by deletion type as 

well as by order of presentation* The cards were scored on the 1620 

with a test scoring program provided by the Cooperative Test Scoring 

Service of the University of Connecticut. The data thus obtained 

gave the number of correct responses for each of the twenty-four tests 

for each student* as well as a discrimination index for each item* 

The cloze answer sheets were hand scored by student assistants to 

* 

determine the number of correct responses* 

These data were then transferred to tabulation sheets along with 
the test scores for both pre- and post— tests* The data were punched on 
cards and processed on the 7040 with a program which produced means* 
variances, standard errors, and correlations of all the variables* A 
second program was used to produce t-tests between each variable for 
each form* The correlation matrix thus produced was 111 x lit* One 
of these was produced for each of the deletion types. 

TWo pilot studies were conducted. These are reported on the 
immediately following pages in Chapters IV and V* 
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CHAPTER IV 



PILOT STUDY I 

Tho doze procedure has boon shewn to produce gains In reading 
comprehension * When three-hundred-word prose selections were given to 
college students* results on comprehension tests shewed that readers 
of cloze passages with ten peroent of the words deleted displayed 
significantly greater grasp of the material than readers of the same 
passages with all the words (Bloomer, 1962), Previous studies have 
also shown that when words are deleted at random, say every nth word, 
there is a difference In the difficulty of replacing words of different 
grammatical classes, e,g,, modifiers, nouns, verbs, prepositions and 
conjunctions, and articles (Rankin, 1958), The present study, then, is 
concern M with the effect upon the immediate comprehension when doze 
deletions are made not at random but uniformly by grammatical class, 

Since the immediate reading comprehension score seems to be 
related to the final gain made after the reader has done a number of 
doze procedure exercises, it is important to determine if some classes 
of words tend to destroy the structure of prose and cause loss of mean- 
ing, and which dements increase the probability of gains in comprehen- 
sion, 

a^geduye. On dune 3rd and 4th, 1964, 1146 fifth and seventh 
grade students of the schools of South Windsor, Connecticut were given 
a single reading passage in seven forms followed by a single comprehen- 
sion test of twelve questions. The prose was the initial seotion of a 
asshort story by author-dramatist William Saroyan. To retain the 
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artistic aims of the writer in the piece of fiction as a whole, the 
prose was edited only slightly to produce a shorter self-oontained 
unit of fiction and to reduce the original readability from 6.00 (on 
a modification of the 9Teakum scale which omitted common contractions 
from consideration) to 5.00. 

The short story is a narrative made up largely of dialogue 
between a twelve-yeair-old boy and an Indian friend who does not speak 
the elided language of the popular prototype Indians of television and 
movies. 

Six of the forms of the passage were modified so as to produce 
an evenly distributed deletion of ten percent of the words. As a 
control, a seventh form was left unmutUated. The six kinds of dele- 
tions were 1 ) random; 2) nouns, proper and common; 3) verbs, exclusive 
of auxiliary or function verbs; 4) specific modifiers^-adjeotives and 
adverbs; 5) prepositions and conjunctions, exclusive of subordinating 
conjunctions; 6) noun determiners— e.g., articles, possessive pronouns, 
words of the type represented in gome, any, few, which precede nouns 
and exercise rx modifying effect over them. All deletions in a given 
form (except the random form) were of a single word class. 

The students were instructed to read the passage filling in any 
missing words on an answer sheet provided, and then to answer the 
questions following the selection on a mark-sense card to be scored by 
machine. The first six of the questions tested comprehension of the 
literal meaning of the passage. The last six were designed to elicit 
inferences logically implicit in the passage. No time limit was imposed. 
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S&flflA?,- Tb* scores on comprehension tests (see Table IX) 
show that readers of passages with noun determiners deleted had a 
greater understanding of the passage than all other groups* in c lud ing 
those who read unmutilated prose* There was a significant loss of 
comprehension among those who read passages with specific modifier 
deletions* The toores on comprehension tests were (from high to low) 

1) noun determiners; 

2) no deletions; 

3) prepositions and conjunctions; 

<0 modifiers; 

5) specific verbs; 

6) random; 

7) nouns* 

Noun determiners were superior to all other classes of deletions and 
to the control group* The no-deletion control group was superior to 
all deletion types excepting noun determiners* In turn, prepositions 
and oonjunotions were superior to modifier! specific verb* . nom 9 . 
and random deletions* 

In the number of exact replacements of deleted words (see Table 
X) scores reveal that prepositions and conjunctions* and noun determ- 
iners* were more regularly replaced than any other class of word* 

Scores on word-replacement tests ranked from high to low as follows s 

1) prepositions and conjunctions; 

2) noun determiners; 

3) random; 

4) verbs; 

5) nouns; 

6) specific modifiers* 

feP , oIa?iop i g* Marked losses in comprehension ooour where the 
structure of the prose and its meaning are broken by deletion of the 
bMie meaning carrier, of the language, i.e., asa*. and 
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Table X 



MEANS AND t*s OP PERCENT GLOZE BLANKS CORRECT 

(All Students 





Random 


Noun 


Verb 


Modifier 


Prep.* 

— Con.1. 


Noun 


I 


63.57 


* 2.75 


55.41 


39.95 


73.80 


70.86 


N 


50 


48 


' 4# 


49 


48 


42 


Random 


- 


6.21 


2.63 


7*29 


3.26 


2.67 


Nouns 


«* 


m 


3.98 


NS 


9.70 


10.00 


Verbs 


m 


m 


- 


5.04 


6.21 


6.13 


Modifiers 


m 


m 


m 


- 


10.95 


11.58 


Prepositions & 
Conjunctions 


- 


- 


m 


- 


me 


NS 


Noun 

Determiners 


















It is apparent that oven In paesagoa of fairly low readability level 

the duplication of lexical and atmotural meaning la not great enough 

& 

to supjOy the oontextual infomatlon necessity to replace these words 
and reoreate the sentences as the same meaningful utterances they were 
before mutilation* Among the words deleted at random, forty percent 
of the words deleted fall Into these three classes, but only nineteen 
different words are presented, eleven of which appear at least one 
other time in the passage* Both the number of words of these three 
classes compared with the total number of words and the number of words 
that appear only onoe (and therefore never appear when deleted) would 
seem to acoount for the gain over the control group* Of the remaining 
sixty percent of the deletions in this form, all but two were function 
words which contribute no lexioal meaning of their own but relate parts 
of utterances* These words, among the commonest in Shglish, are dictated 
by the grammatical relationships of the sentence signals which are 
regularly duplicated by the morphology and syntactic patterns of the 
language* Significantly, one-fourth of the structure words deleted 
were noun determiners, which may acoount for the perceptible gain 
among readers with random deletions* Among the selections with verbs 
deleted, the missing words are all among the commonest four thousand 
words in the language, according to the Thorndike table, and forty of 
the sixty are among the commonest two thousand words* There are, in 
fact, only thirty-three different verbs represented in the deletion, 
which points to a lexioal duplication of significant proportion* Th is 
would aooount for the lack of perceptible loss among readers confronted 
by passages with missing verbs* 



Thu gain among readers with noun drtev*%dn<ir8 delated can be 
explained in a number of ways 9 and aeveral factors may well be oper- 
ating simultaneously to produoe the increase* 

1 • Focus, Removal of function words where syntax and morphology 
already signal the grauoatical relationships in a sentenoe may tend to 
focus the attention of the reader on the larger meaningful units of the 
passage being read* This would be peurtioularly true of noun determiners, 
the sequences of which are governed not only by the sentenoe in which 
they occur but by the sentences (and paragraphs) which precede them* 
Moreover, if we grant Benjamin Lee Whorf 9 s assertion that the bulk of 
the abstract words in English are nominative, the increased reader atten- 
tion to words following marked deletions would seem to be a focusing of 
the perceptive powers on the individual dements Where gains and losses 
in meaning occur* This would be borne out by the lower scores following 
noun ddetions* Since the other body of function words (prepositions 
and conjunctions) would also tend to produoe structure- centered reading 
as well and yet not concentrate attention on nominative elements, the 
difference in comprehend on soores between classes of function words 
would tend to confirm the hypotheds that nominatives carry the burden 
of abstract meaning* Where the larger meaningful structures are 
shattered by ddetion of words ohieh cannot be supplied, e*g* c specific 
modifiers^ it is likely that the reader shifts his attention to the 
less meaningful units* Since ddetion of prepodtions and conjunctions 
dso produced higher oomprehondon soores than all other forms of 
deletions exoept noun determiners it Is plain that a structure- oentered 
reader is more likely to read with oomprehendon* 



2 * Novelty. The requirement that the reader substitute as accur- 
ately as possible the missing word provokes in him a response to a novel 
reading experience* and when the re% :.»r is reasonably confident that he 
has correctly replaced the missing word— highly probable where he is 
^pplylug function words dictated by the grammar of his language— thi s 
may tend to relieve other anxieties he may have when subjected to the 
reading-testing ordeal. Correspondingly* if he is unable to assure himself 
that he has replaced the words correctly (as in oases whore he must guess 
at specific modifiers) his comprehension of the material may be affected 
adversely by a frustration factor. 

3« Repetition* Repeated reading required to supply missing words, 
pa >ioularly where the words are governed by structural principles not 
always apparent in the immediate sentence* would appear to be a form of 
reinforcement of the material. Repeated reading where the missing link is 
incomprehensible would tend to break down the unity of the material under 
scrutiny, reinforcing the baffling aspects of the material and solidifying 
in the reader's mind only that aspects of the material are beyond his grasp. 

ft*® extra time required to supply missing words in the 
text would increase the exposure of the reader to the material, allowing 
more time for the infonoation to crystallise, and the neoessity of stopping 
regularly to supply the missing word® would oonstitute a regular break, not 

ox fared by paragraphing and punctuation* thus providing a method of deliber- 

* 

ately laminating the information in layers which are fairly uniform. The 
extra time where the words are not readily supplied would allow the reader to 
reinforce only his knowledge that there are significant gaps in his under- 
standing of the material. He may* in fact* spend so much time attempting 



42 



to place the word that he loses what fragments of Information are pres- 
ented to him. 

It is apparent that if structural relationships within an utterance 
are expressed simultaneously by two or more grammatical devices, one or 
more of these structural pointers may be unnecessary* Morphology and 
syntax by themselves, however, may be insufficient to convey the function 
of a word and thus the relationships among several words* When too many 
of the fUnotion words are missing the result is likely to be ambiguous if 
not meaningless* Professor Fries 3 now classic example "Ship sails today" 
indicates one result of too few grammatical signals* 

Since lexical redundancy varies markedly from one prose sample to 
another, we may not infer that deletion of verbs and nouns produces no 
loss in comprehension in all kinds of prose* And it may well be that the 
number of different words within a given class when several classes are 
deleted at random will affect the reader 1 s comprehension of the passage* 

The present study in addition to particularizing the general 
findings of the earlier experiment with three-hundred^word passages 
allowed the experimenters to test the effects of the longer passages, 
using a fairly large number of words without increasing the percentage 
of deletions* The longer passages are perhaps more realistic appraisals 
of general demands made of readers* A future experiment will disperse tho 
deletions to an even greater extent among various kinds and m^des of 
literature, involving subliteral, e*g* symbolic or ironic, meanings. 
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CHAPTER V 



PILOT STUD! II 

■£It is probable that when materials are used to elicit close 
responses, all of which are function words, the subscript line marking 
a missing word is a variable component of the reading processes. Function 
words, or "functors," are by definition members of a body of fixed units 
in language which must be memorized because their meanings are chiefly 
grammatical. They contribute relatively little lexical meaning to an 
utterance, and stand as markers of the interrelationships of other con- 
stituents in an ordered stream of words. Because this small, fixed body 
of words is committed in constructions by one who learns English and 
because such words compose about one third of the bulk of the language as 
it is spoken, the presence of any marker in many constructions is likely 
to provide the probable link in grammar. 

It would also seem that the requirement that the reader make an 
actual replacement of the word by writing it in should affect the reading 
process by interfering with the continuum of a prose passage, perhaps 
producing considerable rereading of individual segments. In the case of 
function words which earlier studies indicate are supplied readily, the 
investigators speculated that perhaps the subscript marker supplied suf- 
ficient syntactic information for the reader to grasp grammatical relation- 
ships and that the regular stops to write in words tended to fra gm ent the 
prose tor the reader. 

To assess the effects of both the subscript marker or underscore 
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and to appraise the effects of requiring a reader to write in his 
doze responses, a reading selection was prepared in several forms 
and assigned at random to 420 seventh and eighth grade students of 
two Manchester, Connecticut schools* 

Procedure . Words deleted were of three dasses, one dass to 
a subject* An evenly dispersed deletion of ten percent of the words 
(all. of a given dass) was obtained by dividing the prose passage into 
ten-word stiches and ohoosing the word of the desired dass as near 
the end of each stich as possible* 

Class one was prepositions and conjunctions, exdusive of 
subordinating conjunctions* 

Class two was noun determiners, also known as "noun markers”, 
such as a, the , some, anv and possessive pronouns in constructions 
such as his book . 

Class three was a body of pronoun substantives which, occurring 
alone, replaced a larger or more specific nominal unit already in 
evidence. 

A single passage was mutilated to prepare doze materials of 
three types, so that the passage appeared in nine mutilated forms and 
an unmutilated control form. 

l^pe A, comprising three sets of deletions (Classes one, two 
and three) was a standard doze form followed by a twelve-question 
comprehension test. Where words were deleted a five-pica subscript 
line appeared with a number corresponding to numbered blanks on an 
attached answer sheet. Subjects were asked to fill in the missing word 
by writing them on the answer sheet, and then to answer the questions 
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without referring to the passage* 

Type B (Classes one, two and three) was the same as A except 
that the answer sheet was eliminated. Subjects were afeked to guess 
the missing words indicated by the subscript lines as they read but 
were not required to write the words and they were asked not to write 
on the test booklet* When they had finished they were to respond to 
the comprehension questions without consulting the reading passage* 

Typ© C (Classes one, two and three) passages were not standard 
doze passages* Ten percent of the running word count asls absent with 
no symbol to indicate that such mutilation had boen made. In Type C, 
form one, for instance, the same propositions were absent from the same 
contexts as in form one of Types A and B, with the difference that no 
subscript line indicated the omissions* Subjects who received Type C 
forms were told nothing of the omission, but instructed only to read 
the passages and to respond to the comprehension questions following 
their reading without consulting the passage again* 

The passage was the same one that had been used earlier on fifth 
and seventh grade students at South Windsor, Connecticut* schools, a 
six-hundrod-word narrative of a twelve-year-old boy and an Indian friend. 
Readability measured by the Yoakum scale was 5*0. The passage was 
colloquial in nature, and the unmarked deletion of function words in 
many places produced pidgin dialect not unlike the ordinary speech of 
motion picture Indians. 

The passage was followed by twelve multiple choice comprehension 
questions which had been protested in the manner described in the 

** 



procedure. Pour hundred and twenty seventh and eighth grade pupils 
were assembled in two sessions in a junior high school for the purpose 
of the experiment. The subjects included all of the children in 
seventh and eighth grade who were present in school on the day of the 
experiment. The subjects were randomly assigned to treatment group 
and to class of deletion. All treatments were given simultaneously 
to all children, thus avoiding any effects of selection of children 
for school class, sex, or intelligence. The pupils were given appro- 
priate written instructions to follow. 

Results , The data for the present experiment were the mean 
number of comprehension questions answered correctly by pupils in the 
various treatment groups. These data are found in Table XL* Bata 
were treated by analysis of variance as shown in Table XU, and indicate 
that ho significant difference was found between either methods or clas- 
ses of deletion. 

The results of the experiment therefore indicate that the sub- 
script line or the requirement to fill in the missing word in cloze 
procedure does not produce a result which is different from reading 
material from which words have been deleted but where there is no 



indication other than syntactic or structural indications of the dele- 
tions, Whatever effects therefore may be found in the doze procedure 
will appear to be not a function of the physical structure or the require* 
ment to make responses, and any short-terra results with non-reinforced 
cloze procedure therefore must be seen as a function of the language 
mutilation in and of itself and the information which is carried through 
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the mutilated language. 

Comparison of this group with a similar population in the main 
group indicates that oach of the three treatment types is superior to 
the control group for preposition and conjunction deletions, and for 
noun determiner deletions. Any of these forms of handling a single 
cloze exercise seems to produce superior comprehension than does straight 
reading or mutilated reading at the seventh grade level, though a 
specific test of this hypothesis would seem warranted. 



Table XI 

MEAN NUMBER OF CORRECT COMPREHENSION QUESTIONS 
BY TREATMENT AND DELETION TYPE 







Treatment Tvne 




Deletion 

Type 


1 

doze 

Non-Answers 


2 

Cloze 

Answers 


3 

Pidgin 

English 


Prepositions and 
Conjunctions 


7,6 


7.9 


8.3 


Noun Determiners 


8,2 


8.3 


8.1 


Pronoun 

Substantives 


7.8 


8.5 


7.8 




Table HI 




ANALYSIS OF VARIANCE OP MEAN COMPREHENSION QUESTIONS 




CORRECT BY TREATMENT AJW DELETION 






Type 

Sum of Sauares 


Mean 

d.f. Sauare 


F 


Rows (Treatments) 


,20 


2 ,1 


1 .03 N.S. 


Columns 

(Deletion Types) 




2 *055 


.56 N.S. 


Residual 


.39 


* .975 




Mai 


t20 


8 





1 
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CHAPTER VI 

THE EFFECTS OF NON-OVERT REINFORCED CLOZE PROCEDURE 
UPON COMPREHENSION BY GRADE AND SEX 

If ono were to follow strictly the tenets of operational 
behaviorism, the process of reading is an extinction process* No overt 
reinforcements are given for reading words as sentences except at early 
grades, unless we consider that such overt behavior as page turning or 
finishing a book is in fact sufficiently reinforcing to sustain the 
prooess, or if some extrinsic award is awaiting the completion of the 
reading. While it is true that some derive considerable reinforcement 
from having finished a book and then reporting to friends or teachers 
that it has bee** read, the vast bulk of reading is not for this purpose. 
While this is an oversimplification of the reinforcement position, it 
points to the necessity for the postulation of a non-overt reinforcement 
system to account for the phenomenon of sustained reading* Self reinforce 
raent is derived from the individuals ability to match the author^ 
language patterns with his own to derive meaning from the printed word 
and to interpret these meanings in view of his own experiences* 

The ability of an individual to derive self-reinforcement from 
reading material is dependent upon the magnitude of the reinforcing 
stimulus, which in turn is directly related to the difficulty of the 
reading material. Where material is too difficult we find that the 
ability of the individual to self-reinforce is lower than the amount of 
energy or frustration that he must exert to read. Alternatively, if 
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the material la too easy* the increment of reinforcement is too small 
and exceeds the extinction ratio* The ability of an individual to 
reinforce himself while reading material is directly related to his 
attention to the material and to his reading comprehension* 

Given this position* therefore* we can deduce that reading com- 
prehension will be relatively low where the material is difficult for 
a reader* In turn* as the relative difficulty decreases* reading com- 
prehension will not increase propor tionately * The area of extreme dif- 
ficulty in reading comprehension has been explored through the develop- 
ment of the readability scales* The area of supreme simplicity* on the 
other hand* remains to be explored* 

The doze procedure has a tendency to make reading materials more 
difficult and as such we should expect responses to the doze procedure 
a dmilar pattern but to give different values at different 
grade levels* Cloze procedure has still a further advantage in that one 
can relatively control the amount of difficulty of doze procedure pas- 
sages by controlling the syntax of the ddeted material* Noun* verb and 
adjective deletions tend to be more difficult* whereas pronouns* conjunc- 
tions and noun determiner deletions tend to be considerably simpler* 

Hence doze procedure offers a flexible tool for investigating the 

limits of reading comprehension in rdation to reading material 
difficulty* 

The present cross-sectional experiment was designed primarily to 
compare boys with girls at the fifth* seventh* ninth and deventh grade 
levds* and the long-term recall of doze procedure materials with dele- 
tions of various syntax against straight reading material and random doze. 
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AH doze materials in the present experiment were ten percent deletions, 
primarily because it is relatively Impossible to use a higher percentage 
of deletions and at the same time specify the syntax of the deletions • 



PROCEDURE 



Twenty-four six-hundred-word passages were prepared. 
Twelve oomprehension questions were prepared for each one. These were 
pretested and the data analysed for item discrimination. Adjusted 
questions or paragraphs were pretested a second time to ensure that the 
items did discriminate. All selections were adjusted to fifth grade 
reading level using the Yoakum formula and all the adjustments were made 
in such a way as not to affect the meaning of the paragraph. Each 
selection was prepared in seven different forms; 



Type 0 
lypo 1 
Type 2 
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Type 
Type 
Type 6 
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Zero deletions 

10# deletions: Random 

10# deletions: Wouns 

10# deletions: Ttrbg (auxiliary verbs not included) 
10# deletions: Adjectives 

and Conjunctions 

» Op deletions: Noun Determiners. 



The sample for the present experiment consisted of fifth, seventh, ninth 
and eleventh grade pupils, totaling 976. from a regional sohool adjacent 
to a large city in upatate New York. 



Each student was given a paokage of six of the selections 
along with the oomprehension queatlons and appropriate answer sheets for 
answering the dote material. Hie envdopes were passed out to the pupil 
at the beginning of the class period on two non-conseeutive days during 
three weeks. At the end of eaoh working period the materials were 
collected from the student and remained with the teachers. The student 
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was instructed to read the selections, to fill in the blanks where 
appropriate, >o answer the questions and to return the material to 
the teacher. The teacher did not grade the material, reducing the 
possibility of overt reinforcement. Each student received six selee- 
tions with the identical deletion type but within ary class deletion 
types and seleotions wero randomly assigned to students to control for 
class differences, 

RESULTS 

Betwean-Grade-Level Comparisons . Table ]fall gives the means and 
variances for each deletion type at each grade level, Reading compre- 
hension scores for Grade 7 are superior to those for Grade 5 beyond the 
.01 level of confidence for all comparisons except for Type 2, noun 
deletions. The difference is not significant although it is in the 
expected direction. The t tests rang® from 2.91 for Type 3# verb dele- 
tions, to 6,8? for Typo 6„ noun determiners. 

The comparisons between Grades 7 and 9 indicate that six of the 
comparisons are in the expected direction and significant beyond the 
*01 level of confidence. The t*s range between 4.07 for random deletions 
to 6,87 for the aero deletion group. The exception to this trend occurs 
in the noun determiner group where the reading comprehension following 
noun determiner deletions for Grade 7 is significantly greater than the 
Grade 9 comprehension for the same deletion type. Grade 7 is signifi- 
cantly superior at the .05 level of confidence: t » 2.44. 




The Grade 9 * Grade 11 comparisons indicate no significant 
difference between the groups except that Grade 11 is significantly 
superior to Grade 9 for the noun determiner group, Type 6 (t~£ 3*31 )• 
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o 

ERIC 



© 



o 

SB 



♦ 

-o 

6 

O 

US 



a 

% 

cu 



g 

© 

-4«rt 



JSS 



St^A j § 

Eh m 

•g > 
§ 

g" | 
3 I 
A 



& 



ot 

1 

O 



•3 

-p 



5 

o 5 



1 

© 

1 



N. r» 

8 w t > 

• • 

r- VTv 



«? S . S . 
v - IA ^ 



g c i 






*s 



• . CM 
sslx CO 



5 

us 



& £ 

• • 

*- C' us 



CM to 3 C- 

l > • • C - 

»" SO l/N «- Cs . IA ♦— 



sO - d * to *- 

to us cn US 0- 

Q * • • • 

»“ IA r» IA vO 



CM 


_ (A 


CM 


I s - 


Oso 


CA 


• 


O • 


• 


US 


t-nO 


US 


Os 


to 


to 


-4* 

• 




CA 

• 


Os 


V- us 


v— 






T-” 


us 


OA 


us 


>o 


sO CM 


sO 


• 


US • 


• 


US 


»" nO 


US 



C ? Qn^O & 



8 ^ N . S ' . . 

OI>A <A ^vO vO 



CM 

S5lH CO 



i 



2 * Os OS O 

f-tO Vf \ OsnO US 

£*- • • ♦ 

»- nO nO (A 



g » 



US CM 



D us 
«" nO 

*• fr - * 4 “ 



S 



• • 
r» f* nO 



& 

*- nO 



(*A o - 

to NO 



C*A us 
CM <*A OA 

to 



Os i— 

CM • • 

CM tO ^ 



CM 

SIX CO 



I 



3 



us 



[S. T— 






& 

CM 



CM 

SIX CO 



3 



8.22 7.02 6.84 7.45 6.87 7.66 

4.81 5.74 4.64 8.09 2.86 4.58 
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It should also bo oomnonted at this point that the noun determiner 
group was not superior to the Grade 7 noun determiner group. Two 
possible explanations offer themselves for this phenomenon. The 
first is that the ninth grade reached the upper celling of the test 
and therefore the eleventh grade oould not be expected to achieve 
higher scores. This, however , appears improbable for several reasons. 
First, the mean number of responses made by eleventh graders was less 
than two-thirds of the number of possible responses. Further, in 
four of the comparisons the responses made by the ninth graders was 
superior to the eleventh graders, indicating that it is possible to 
get higher scores, albeit these differences were not significant. 

It is important to bear in mind that the material which was 
presented to the eleventh grade pupils, the same as the other material, 
was at fifth grade level. Hence its level of difficulty for these 
pupils would be relatively low. As the level of difficulty becomes 
extremely simple, the value of making a correct response is lost, and 
hence the individual is operating at a reduced motivation. The require- 
ment to attend carefully to the material in order to get a correct 
response is dissipated where t > material itself is simple, and hence 
the strength of a self-reinforcement would be dissipated, along with 
the attention factor. The quality of the self-reinforcement, therefore, 
is dependent upon some level of difficulty. This is further substan- 
tiated by the fact that the ninth graders, when dealing with the type 
6 noun determiners where the probability of a correct response is .33, 
did relatively lees well than the seventh graders. We can assume that 
the ninth graders would find this material exceptionally easy and there- 
fore without motivating value which would overcome the amount of energy 
o 
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output required to read the material and answer the doze responses* 

Ihe seveiith graders, on the other hand, find this level of difficulty 
to be highly motivating. This is confirmed not only in the present 
experiment but also in the preceding pilot studies. These data oon- 
firm the hypothesis that it is possible to have comprehension materials 

whioh are too simple for a particular group to achieve maximum compre- 
hension. 

Type Comparisons. In general, at all grades, Type 2 (noun 
deletions), Type 4 (adjective deletions), and Type 3 (verb deletions) 
tend to be inferior to the control group. Type 0, and to Type 1 (random 

deletions), Type 5 (preposition and con juneUon deletions, and Type 6 
(noun determiner deletions). 

Selections containing Type 0, Type 5 and Type 6 also contain 
the greatest amount of information and present the leapt difficulty 
for the students. 

Recall of Comprehension M aterials . The recall test was com- 
prised of thirty-six questions. Six of the twelve comprehension ques- 
tions which followed the selections were randomly chosen for the com- 
prehension test. Each student wae assigned a form of the test n - 
surate with the selections whioh he had previously read. The rec all 
test was given ten weeks following the last of the selections. Hie 
means and varlanoes are presented in Table XIV. The rank order for 
recall comprehension indicates that Type 6 (noun determiners) is first, 
followed by Type 5 (prepositions and conjunctions). Type 2 ( nouns ), 

Typo 3 (verbs). Type k (adjectives). Type 0 (no deletions) and Type 1 
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(random deletions). Types 5, 6, and 2 were each found to be signifi- 
cantly superior to the Type 1 random deletions for reoall comprehension: 

t 2,1 « 2.10| t 6,1 « 2.97; and t 7,1 55 2.30. No other significant dif- 
ferences were found. 

It should be noted here that the type 5 and Type 6 were, for 
all grade levels, typically among the highest scores on the initial 
comprehension test, whereas Type 2 was quite typically one of the 
lowest. Furthermore, the Type 0 comprehension exercise material was 
again among the highest, and on the recall was among the lowest. This 
raises a question as to the relationship between the immediate learning 
and the recall of infomation. Results from learning experiments show 
that the greater the initial learning, the greater the recall. However, 
under present experimental conditions each subject was resp onding to a 
single exposure. Following the reasoning in the field that recall is 
related to initial learning, one would expect relatively high correla- 
tions between immediate comprehension test scores and reoall soores. 

This proves not to be the ease in the present instance. The correlations 
range from a +.?6 to a -.29 with an average determined by z-trans'orma- 
tion of .20. Further, it would seem highly probable that the number of 
correct closures should relate to immediate comprehension test scores. 

These range from .76 to -.11 with an average determined by z-transfoma- 
tions of .20. Further, there appears to be no relationship between the 
number of correct closures and the number of reoall comprehension 
questions answered. Correlations here range from +.56 to -.92. with an 
average of -.01 9. The indication here is dear, doze procedure itself 
seems to be testing something different from the usual type of comprehension 
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Question* This result would ipppost othor findings such as 'those by 
Weaver and Kingston^ in the same area. Further, initial reading 
comprehension test scores, whether close procedure of not, are testing 
something different from final recall procedures, which would indicate 
perhaps that if one is concerned with the long-term remembering of 
materials, the procedure of paragraph comprehension exercises which we 
customarily use to teach reading comprehension is not adequate. In 
essence the long-term recall of these materials appears to be a quite 
different stall from the reading comprehension skills which are taught 
via paragraph comprehension techniques. 

3ex Comparisons by Grade « Tables XV and XVI show the means and 
variances for boys and girls by grade level. An analysis of variance 
indicated significant differences between grades but no significant 
difference between the sexes. The data show that in thirteen instances 
out of twenty-eight, the boys achieved superior mean scores. This 
difference was significant in two instances. The comparisons between 
boys and girls at the ninth , grade level shows the boys to be significant- 
ly superior (Type 0, t * 3.98, Type 1 , t * 3.69). The girls appeared to 
be superior in fewer instances as the grade level advanced. Fifth grade 
girls were superior to the boys in three instances: Type 1 , ra ndom 
deletions (t « Z. 06); aype 3, verbs (t • 3.15); and Type 6, noun deter- 
miners (t * 2.65). At the seventh grade level girls were significantly 
superior to boys for Type 2, nouns and Typo 6, noun determiners: (t = 

1.96 and 2.70 respectively). At the ninth grade level the girls were 
superior to the boys in one instance, noun determiners (t « 3 . 50 ); and 
at the eleventh grade level girls were superior to boys in one instance, 
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Table XV (continued) 
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random doae (t * 4.70). Data auggaat that girls, therefore, appear 
to ba more responsive to th* noun determiners than do boy a. In that 
in thrae UTstanoea out of four thay war* algnlfioantly superior; and 
tha data also suggest that tha difference between boys and glrla, which 
nay ba prasant In tha aarly years, saws to disappear aa tha ohildran 

get older* 

QLrls at the seventh and ninth grade levels seam to be better able 
to reinforce thwselves ftoa thair language patterns than are boys. A 
oonparison between tha gradas shows that ninth grads boys are superior 
to eleventh grade boya in Type 1 (t - 2.96) and inferior to eleventh 
grad* boys in Type 6 comprehension (t - 2.36). The mean score for boya 
in Grade 9 la superior to tha naan score for boya in Grad* 11 in five 
of the six eonpartaona. This may b* a function of the differential 
approach to language, materials of boya and glrla, and indicate, as do- 
th* other data, tha tendency for younger boy* to be self-reinforoed 
by oorraot response* in dose procedure materials la sonsMhat laea than 

that of girls. 

1KSCU33I0H 

The data the hypothasis that essentially higher grad* 

pupilc oonfronted with material of axtrw* sl mpll oity, approximately 
six reading levels below thsir mean ability, oould not derive suffloient 
self-reinforosoent from reading the materl.CL or ooopletlng th* dose 
axsrds*s to oonprehend at a lead better than that of th* ninth grad*. 

ssx oomparlaons indioat* that, as th* material gets relatively 
more simple, th* differences between th* same saws to disappear and tha 
tendency for boya* performance to be superior over girls increases. This 
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suggeirte testing the hypothesis that teaching boys at lower levels with 
material which is somewhat simpler may erase the traditional difference 
between boys and girls in reading ability. Type 0, 5 and 6 materials 
confirm the hypothesis of self -rei nf or cement • It remains, however, to 
determine why the other deletion types do not follow this pattern. The 
due to this dilemma is found in the decreased performance of the ninth 
graders on Type 6 (noun determiners). 

The fact that relative comprehension decreases with simple 
material gives rise to a requirement to examine school materials, 
particularly at the first grade level, to determine if the simplicity 
at that level is not debilitating to reading progress. 

The data also suggest extreme care with the pre-reading survey 

* 

techniques. The effect of the pre-reading survey may be to reduce the 
value of the reading material in the eyes of the individual such that he 
will not then comprehend the material. Obviously this caution would not 
hold where the material is new to the reader. However where the material 
is familiar, pre-reading may oause him to pre-Judge and to underestimate 
t’k* *alue of the material. Under these conditions survey techniques w il l 
reduce rather than increase reading comprehension. 

The experiments suggest caution in the interpretation of certain 
test results. It is common practice in the building of standardised teats 
to extrapolate the performances of the extremes of the standardisation 
group from data found at the middle of the standardisation range. This 
practice may have the net effect of making completely unreliable scores 
at the upper extremes of the range. At the upper limits of a standard- 
isation group where self-reinforcement is relatively low, reading test 
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scores would be more dependent upon motivation than upon actual reading 
ability. Working at the extremes of the range for reading tests 
standardised in the above manner, pre-test arud post-test differences 
would tend to be indicated less often than they occur. Further, a 
standardised comparison of one group with another would not reflect a 
true index of differences between the groups. Hanking individuals 
within the class would be more likely to rest on motivational variables 
rather than upon reading; ability*. This may aooount for the fact that 
many studies of college reading programs have not found significant 
reading comprehension differences. The data lend credenoe to the test- 
makers adnonition that one should select a test where the group to be 
tested is close to the middle of the range of the soores. 

The data also suggest further exploration of the notion that 

♦ 

dose procedure is measuring a comprehension skill whioh is different 
from the traditional paragraph, question technique. The highly variable 
correlations between dote procedure and reading comprehension indicate 
that serious exploration may reveal new and useful dimensions for the 
measurement of reading skills 



CHAPTER VII 



THE EFFECTS OF NON~REXNFQRCED CLOZE PROCEDURE 
UPON THE AHELJCTY TO HAKE CLOSURES 

The previous chapter found a differential in the ability to 
make oloeuree baaed upon the syntax* and indicated a eelf«*reinforoing 
factor in operation dur i ng a abort sequence of dose exercises* The 
duration of a long series of dose exercises where no overt reinforce* 
meat is present takes the aspects of extinction trials. We oan 
expect therefore some effects in terns of the ability to make dosures 
during a prolonged sequence of non-reinforoed dose exercises* 

In an earlier study* Bloomer and Heitsaan found an inequality 
in the ability of individuals to make dosures of varying syntaxes* 

The question arises* particularly when dealing with self -reinforcement * 
ss to whether this difference in difficulty will in turn effect vari- 
ables suoh as the ability to w a ke dosures* end reading comprahension* 
and more particularly will practice with dose procedure materials of 
a particular syntax Increase the ability of an indi vidual to mk t 
dosures of that same syntax* 

With the postulation in the previous chapters that an individual 
is able to reinforce himself by producing language patterns and matching 
those with the language patterns which he has already learned* one would 
expect seme increase in the ability to perform dose material subsequent 
to practice with non-relnforoed dose exsrdses* Comparisons* therefore* 
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between the control group (that is, those persons who performed the 
exercises with no closures) and close procedure experimental groups 
who performed close procedures of various syntactical deletions 

lndio*te » superiority on the pert of the experimental groups to per- 
form oloxe prooedure materials. 

Here again we would expeat that some of the syntaot deletion 
types would not generate^suffident feedbaek for individuals to learn. 
Others where the probability of reinforcement le sufficiently high 
would not prodnoe sufficient reinforoement for an individual to 
olosures. therefore one would hypothesise that the relatively more 
difficult alone prooedure exercises end the relatively more simple 
dose prooedure exerdses would not produce an Increase In the ability 
to males olosures. However, those of moderate difficulty should Increase 
the ability of Individuals to make olosures. 

•**•«*» of extinction, however, should result in s shift 
in perodvsd difficulty such that all materials should appear more 
difficult, Therefore under conditions of the present experiment one 
«ould hypothesise that if an increase in the ability to make olosures 

occurs it will be most likely found in the easier samples (i.e., 
function words). 
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With the postulation in the previous chapters that an individual 
is able to reinforce himself by producing language patterns and match- 
ing those with the language patterns which he has already learned, (me 
would expect some increase in the ability to perform close material 
subsequent to practice with non-reinforoed dose exercises. Compari- 
sons, therefore, between the control group (those person who performed 
the exercises with no dosures}' and dose prooedure experimental groups 
*ho performed dose procedures of various syntactical deletions should 
indicate a superiority on the part of the experimental groups to perform 
dose procedure materials. 

Here again we would expect that some of the syntaot deletion 
types would not generate sufficient feedback for individuals to learn. 
Others where the probability of reinforcement is sufficiently high 
would not produce sufficient reinforcement for an individual to make 
dosures. Therefore one would hypothesise that the relatively more 
difficult dose prooedure exercises and the relatively more simple dose 
procedure exercises would not produoe an increase in the ability to make 
dosures. However, those of moderate difficulty should increase the 
ability of individuals to make dosures. 
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The materials for the present study consisted of 

% 

twenty-four short articles six hundred words in length. The artioles 
were selected such that a complete story was told. Material was all 
fiotion, Twelve comprehension questions were written for each of the 
articles* The twelve comprehension questions were pretested to dis- 
criminate between the top and the bottom students in the pre-test 
group, The reading level of the artioles was between 5,0 and 6.0 on 

the Yoakum formula. The articles were then adapted to dose procedure 
in the following manners 

Jype 0. The Control Groups sero deletions 
Type 1, Ten peroent random deletions: 

Type 2. Van peroent noun deletions 
Type 3. Ten peroent verb deletions 

Ten percent modifier deletions 
jy 5* percent preposition and conjunction deletions 
iyp© p* Ten peroent noun determiner deletions. 

Deletions were done in the usual way. The initial word to be 

deleted was selected at random and every tenth word thereafter was 

deleted. Where specific syntax was involved the nearest word of the 

appropriate syntax to the tenth word was deleted. Other grammatical 

categories were omitted since they did not include ten percent of the 
running words. 



§83S}&& ®*e sample consisted of 1h9 ninth grade pupils of a 

•utourbMi osntral school in up-sxate New York. The pupil, were randomly 
assigned to deletion type. Bach pupil received exercises of only a 
•ingle deletion type. The material was worked in class on alternate 
d * y, « two exercises P»r week, for twelve weeks. 
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treatment groups were to be found in each class to control for class 
assignment variables* The students were assigned to four different 
orders of presentation of the exercises to control for order of 
presentation variables* The students wrote the correct responses to 
the closures on a separate answer sheet* The subsequent reading compre- 
hension questions were marked on IBM sense cards for processing 
directly through the computer. There were no significant differences 
between various orders of presentation, hence all orders of presentation 
were treated together. There were no significant differences between 
boys and girls, hence sexes were treated together* 

The pupils were given a pre-test* Subsequent to the experiment 
the pupils were given a final close test consisting of a thousand-word 
passage with twenty words deleted from each of the five syntactic types 
listed above* This test allowed the comparison of the ability of the 
various groups to make closures, as well as the ability to make closures 
of a specific syntactic type. The pupils were also given the California 
Test of Mental Maturity as a part of the post-test battexy* 

The , Test , * A dose procedure test was administered to the groups 
one and one-half weeks subsequent to the termination of the experiment. 
The test itself was a doze procedure exercise one thousand words in 
length with a ten percent random deletion* Twenty of the deletions 
were of each of the five syntactic types used in the study; that is: 
nouns, verbs, modifiers, prepositions and conjunctions, and noun 
determiners* The syntax of the ddetion types was randomly ordered 
within the test, and where the tenth word was not of the assigned 
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deletion type, the nearest word of the appropriate syntax was deleted. 

The subjects w ere required to fill in the missing words. The tests 
were scored for the number of correct responses at each deletion type 
as well as the total number of correct responses* In the present study 
the zero deletion type served as a control group, having read and 
answered the comprehension questions for all the material but having 
had no practice In doze procedure* 

RESULTS 

Closures by Exercise. Table XVI gives the mean number of correct 
closures for Exercises 1 , 10 and 22 by deletion type* Different numbers 
of correct responses were made for the varying deletion types. These 
seem to fall into three groups* Random deletions, noun deletions, and 
vexb deletions were approximately equal. On the other hand, very few 
of the modifier deletions were answered correctly, while both preposi- 
tions and conjunctions, and noun determiners, the function words, tended 
to be answered correctly by a relatively large proportion of the students. 
The number of correct dosures made on Exerdse 10 was smaller than the 
number of dosures for Exerdse 1 , excepting the noun determiners. 
Similarly, for Exerdse 22 the number of correct dosures for all cate- 
gories except modifiers and noun determiners was smaller than the number 
of correct dosures for Exerdse 1 , indicating that in general the 
individuals tended to do less well as the sequence with non-overt 
reinforced doze procedure exerdses continued* Table XVU indicates 
this tendency was not significant for the group receiving random 
deletions* There were no significant differences for modifiers, or 
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Table XVI 

MEAN NUMBER OF CORRECT CLOSURES FOR EXERCISES 1, 10 AND 22* 

BY DELETION TYPE 

Brerolae* 



Deletion Type 


N, 


1 




10 ; 


22 


X 


SD 


—J 

X 


. SD 


X 


SD 


2 


Random 


14 


25.94 


5.32 


20.13 


9.96 


24.73 


7.34 


3 


Noun 


12 


22.36 


6.57 


13.92 


9.66 


13.90 


5.43 


4 


Verb 


11 


22.67 


8.01 


14.56 


8.04 


14.00 


9.34 


3 


Modifier 


15 


14.26 


3.66 


13.00 


6.85 


15.66 


6.49 


6 


Preposition and 


















Conjunction 


25 


37.56 


6.62 


36.88 


8.17 


35.52 


8.49 


7 

*/ 


Noun Determiners 


21 


37.10 


11.13 


47.05 


5.93 


41.44 


13.49 



Table XVII 

*t' TEST COMPARISON OF NUMBER OF CORRECT CLOSURES 
IN EXERCISE 1 WITH EXERCISES 10 AND 22 



Deletion Type 




Comparison Exercise 




10 


22 


1 Random 

2 Nouns 

3 Verbs 

4 Modifiers 

5 Prepositions and Conjunctions 

6 Noun Determiners 


NS 

3.62 

2.37 

NS 

NS 

3-63 


NS 

4.97 

2.32 

NS 

NS 

NS 
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prepositions and conjunctions, and noun determiners were reproduced 
significantly better in Ebcordse 10* However, this same condition 
did not hold for £xerdse 22. 

The data show the differential effects of rate of self* 
reinforcement upon extinction* The random material, which has variable 
difficulty and thus approaches random Internal reinforcement , does not 
suffer the effects of extinction* The function words where the rate 
of reinforcement Is relatively higher similarly show no extinction 
effects* The noun determiners, where the actual rate of reinforcement 
is relatively high and because most of the words are interchangeable 
giving a still higher rate of self-reinf or cement , shows an increase 
similar to that found in the pilot studies with this same deletion 
category with seventh grade pupils* The increase appears to be 
affected by the extinction process in Sxerdse 22* 

In interpreting these data it should be cautioned that the 
subjects for whom data is presented are a select group* All pu pil s 
who had not completed at least eighteen oxer dees were deleted from 
the population for analysis* The group which was not treated are 
presumably those children who suffered the effects of extinction to 
the greatest extent* It should be noted that the N for each deletion 
type in the present study, as oompared with the original N given In 
Chapter II, reflects in all instances a marked drop in number of 
subjects and reflects the relative difficulty of the deletion types. 

The lack of a significant drop in the modifier deletion group 
may be seen as a function of this selection prooess, as well as the 
fact that a relatively wide variety of inoorreot modifiers will still 
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bo acceptable and honoo self-reinforcing. While the specific modifier 

ia difficult to fill in, an individual subject is more likely to fill 

in one which satisfies all the syntactic dues than is the case with 

nouns or verbs* Hence the rate of self-reinf orcement , as with noun 

determiners* is greater than the objective rate indicated by the 
scoring. 

£Lp_gurea by Deletion Type. Tables XV3II,XIX,and XX give the 
significant *t» rations between deletion types for Exercises 1, 10 
and 22 respectively. 

For Exercise 1 , the random deletion group was significantly 
superior to riouns, Verbs and modifiers in the number of correct 
deletions. Deletion Type 5 (prepositions and conjunctions) and 
Deletion Type 6 (noun determiners) were superior to all other types 
in the number of correct closures. 

For both exercises 10 and 22, the deletion types 5 and 6 are 
significantly superior to all other types. Random deletions (Type 1 ) 
shows the effects of limited extinction in that it is not significantly 
superior to noun or verb deletions. 

The final dose test wa* administered one and one-half weeks 
subsequent to the last dose exercise, which allowed time for any 
spontaneous re oo very to occur. The No Deletion group was induded in 
this testing to act as a control comparison group for those trained 
upon dose procedure materials. 

Table XXI gives the mean and standard deviation of the correct 
responses for the final dose test. The mean number of co r rec t 
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*t' RATIOS BETWEEN NUMBER 0 ? CORRECT 






CLOSURES FOR EXERCISE 1 ox DELETION TYPE 




Deletion Type 




Deletion Type 






N 


1 


2 3 4 


5 


6 


1 




3.68 2.81 3.12 


-3.81 


-4.31 


2 






-8.77 


- 7.62 


3 






— 6.01 


-6*17 


4 

e 






-7.70 


•6*99 



Table HI 

SIGNIFICANT •t* RATIOS BETWEEN NUMBER OF CORRECT 
CLOSURES FOR EXERCISE 10 BY DELETION TYPE 




Deletion Type Deletion Type 






H 1.2 3 4 


5 


6 


1 2.24 


-5.89 


-9.54 


2 1 


-7.07 


-10.73 


3 


-7.07 


- 10.86 


4 


-9.23 


-14.30 


5 




- 4.78 



Table XX 

SIGNIFICANT >t* RATIOS BETWEEN NUMBER CP CORRECT 
CLOSURES FOR EXERCISE 22 BT DELETION TYPE 

V ,, j I 1 


Deletion Type 


■ Deletion Type 






N 


j 

to 

4? 


5 


6 


1 


7.15 


-6.19 


-4.03 


2 


3.69 


•6*39 


-4.70 


3 

i a 


2.97 


-5.oo 


-4.00 


4 

5 




—14*32 


- 8.76 
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responses for the experimental groups wars numerically greater than 
ths oontrol group in all hut four instances. Two of these instances 
occurred in the group which had noun deletions* and one each for verbs 
and modifiers. The comparison of the total number of responses for 
the experimental and oontrol groups again shows in each instance the 
number of oorrect responses in the experimental group was numerically 
greater than for the oontrol group. 

It 1 Test C aparisons 

fortTfrrntal Group 1. Experimental Group 1 which was 
trained with random closures showed a significant superiority to the 
control group in total mmber of responses. The contribution to the 
superiority lay primarily in Deletion type 6* noun determiners* and 
Deletion type 5i prepositions and conjunctions* indicating that the 
major learning of individuals who performed non-rsinforced random 
closures is in the area of the function words, 

Bsperlmental Group 2, For pupils trained on noun deletions* 

no significant differences occurred between that group and the oontrol 
group* 

BiMrtflfflfal ®*e group that was trained on verb 

0 

deletions was significantly superior to the oontrol group* as would be 
anticipated from the hypothesis. This same group approached signifi- 
cance* t « 1 ,96* for the number of noun determiners oorreotly answered 
in the test. However* there were no other significant comparisons. 

Experimental f*> The experimental group who practiced 

with modifier deletions performed signifioantly better for total number 
of responses on the final test* as well as for type 6 (noun determiners) 



66 * 







69 



and Type 5 (propositions and oon junctions) , and Type 3 (verbs)* 

Hero again the ability to perform better with function words contributes 
to the major portion of the superiority of this group, but included also 
is the ability of this group to fill in missing verba* 

Group 5 * The group whioh had been trained on 
prepositions and conjunctions was significantly superior to .the control 
group for the total number of responses for type 6 (noun determiners); 
Type 5 (propositions and conjunctions} and for Type 3 (verbs)* In 
addition, this group approaohed significance (t * 1*67) for Type 4 
(modifier deletions) and for Type 2 f noun deletions (t = 1*65)* If a 
one-tailed test were used this group would be significantly superior 
to the oontrol group for modifier deletions* 

Snaerima ntal Group 6* Experimental Group 6, trained on 
noun determiner deletions, had the highest probability of oorrect 
response, as would be expected by the self-reinforcement hypothesis* 

This group did not do as well as those groups with the moderately 
difficult deletion types* Experimental Group 6 approached significance 
in the total number of oorrect responses (t ® 1*70) and with the one- 
tailed test this would be significant beyond the *05 level of confidence. 

The data indicate that whether or not the reading of an 
Individual is affected, his ability to fill in materials of differing 
syntax forms oan be improved by the use of non-reinforced dose materials 
of modifier or preposition and conjunction deletion types* 
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Th© use of doze procedure, particularly of two varieties, may 
prove to be of considerable significance in teaching individuals 
grammar and syntax rules, albeit these rules represent a relatively 
ana H percentage of the number of total areas* The effect of cloze 
procedure is therefore to teach the individual to fill in the specific 
oorreot word rather than a word of appropriate syntax but relatively in- 
correct# In this sense the use of non— overt cloze procedure tends to 
conventionalize the language of the individual# Using non-reinforced 
cloze procedure and comprehension questions related to it, individuals 
given a sustained series tend to do somewhat worse as time progresses. 

One would expect that this is fundamentally the result of extinction 
with the close procedure itself as there is no apparent effect of the 
prolonged series of cloze procedure upon the comprehension of individuals. 
In essence individuals therefore tend to get saturated with non-reinforced 
doze procedure and this saturation appears to show up before ten of 
these have been completed# The question remains whether or not a simi- 
lar situation occurs with reinforced dose procedure# Studies with 
reinforoed dose procedure and motivation have indicated that such a 
saturation does not occur and that different results are obtained in 

<9 

terns of the reading oomprehenalon of individuals who work with random 
close material in a graded sequence; they do get better at reading. 

BISCtJSSION 

Ihe results of the study suggest that there is a differential in 
the capacity of dose procedure exercises of varying syntactic deletions 
tp produoe an inerease in the ability to perform dose procedure material. 
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Verb deletions or noun determiner deletions both act rather specifi- 
cally to increase the ability of an individual to fill in verbs or 
noun determiners, respectively* When one considers modifier deletions 
or preposition and conjunction deletions, the ability to make closures 
apparently generalises to other syntactic types. This is particularly 
true for preposition and conjunction deletions which indicate Use of' 
the non— reinforced technique the most effective way to teach closures. 

It is also significant to note that for Type 2 deletions, no 
increase in the ability to make closures occurs. This would indicate 
that f ill i n g in noun deletions may involve a somewhat different process. 
Typically when one is doing close procedures with noun deletions, one 
may merely substitute for a noun blank the concept thing, place, or 
object in general rather than a specific. The meaning of the material 
is not neoessarlly therefore contingent upon the specific noun name but 
is relatively blear when pronouns or merely one of the noun concepts is 
substituted. Thus the individual who has nouns to fill in has funda- 
mentally a ready substitute set of concepts and does not necessarily 
havr to attend sharply to the meaning in order to figure out what that 
meaning would be. On the other hand, the syntactic rules for a specific 
noun or verb are more restrictive in the close situation than for other 
parts of speech. The individual given verbal close procedure material 
still has a generic notion of action as a possibility for filling in the 
veib. However, the actions tend to be more specific and to relate much 
more to the meaning of the paragraph than do th« nouns. The verb is 
limited somewhat by the noun as well as by the structure of the rest of 
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the material* One must therefore be cognizant of the possible verb 
action of any noun in order to make sense of the close procedure 
material. A certain larger amount of focus is therefore required if 
&n individual is to fill in verbs. He must be cognizant at least of 
the noun about which he is talking. Modifiers are the most difficult 
to fill in simply because a larger range of modifiers will fit any 
particular modification situation and there is no broad overall concept 
which may be substituted. While modifiers do not neoessarily carry 
meanings in a cause and effect situation they none— the— less carry 
a large proportion of the more subtle meaning of the language and are 
the words which tend to differentiate a particular passage from others 
of a s im i l ar nature. To fill in a modifier, therefore, an individual 
must be cognizant of the nouns and of the verbs, and then must choose 
ffcom a relatively large number of modifiers the appropriate modifier to 
fit the particular noun-verb relationship as well as the intention of 
the meaning of the paragraph. 

Much the same is true of prepositions and conjunctions. They 
tend also to function to modify the meaning of the material and hence 
require more focus on the part of the pupil than do some others. It is 
true that prepositions and conjunctions in some measure involve generic 
concepts but in each instance there are typically several of these 
whioh may be chosen, and it is sometimes difficult to determine which 
one would be correctly applied to any particular situation. Preposi- 
tions and conjunctions also tend to make relatively simple clo sures in 
that the probability of deriving a correct closure is relatively high. 
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Ihe noun determiner* typically seem to be of relatively little 
importance as their inclusion or exclusion affects the meaning of the 
material to a relatively mail degree. Typically again the probability 
of a correct response is extremely h* e h . 

At the present time with the emphasis of linguists upon the 
fact that children have developed syntactic sequences prior to school, 
the problem the school faces is often one of changing these syntactic 
patterns to a more standard form. Children, even those from hemes of de- 
viantt linguistic patterns, tend to be aware of standard patterns 
through outside contacts and mass media. Standard speech is thus a 
Amotion primarily of differential reinforcement of the standard over 
the deviant pattern. Since most individuals develop several linguist! o 
patterns to function in the various roles they play, the aohool'a 
concern is with ensuring that the more formal standard patterns are 
available to the individual for situations where these patterns are 
dvantageoue to the individual. The dose procedure may have some 
utility in the teaching of syntactic patterns. It presents a series 
of language profaleme in sequence which would help the child to learn 
to develop language pattern* which, at least for himself, are self, 
reinforcing. Close procedure therefore would allow a teacher the tech- 
nique for teaching the child to produce satisfactory language sequences 
in a relatively controlled situation. The use of the concept of fading 
in a fashion similar to that described by Skinner, where one would perhaps 
increase the percentage of deletions required in material, would teach 
the individual to produce longer and longer appropriate langn.g. 

sequences, whereas the initial material would still offer a framework 
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in which this work oould be done. Thus the children would be taught 
to fill in words of appropriate syntax and meaning in relatively 
logical sequences. It would appear that it would be superior to use 
reinforced dose procedure materials rather than non-roinforoed close 
procedure as the effectiveness of reinforcement in shaping would then 
be taken advantage of, Furthermore, an individual should function at 
some specific level of difficulty of closure or syntactical deletion 
until some criterion were reached, at which point the individual would 
move to more difficult material. It is also dear that grading should 
not be done for the specific word which the author induded, but rather 
that grading should be done on the basis of words of appropriate mean- 
ing, with preference given for those words which are unusual or rela- 
tively less frequent. This would be more and more true as the percent- 
age of deletion increased and the skill of the pupil became greater. 

The present experiment would suggest that the use of preposition and 
conjunction ddetions at least at the outset of the teaching would be 
superior. However, it must be remembered that this is a function of the 
non— reinforced situation in which the material was presented. In a 
reinforced situation some other form or forms of deletion would appear 
to be superior. Certainly if a child were having difficulty with words 
of a particular syntactic type, doze procedure materials, excluding 
words of this particular type, would serve as a training device for this 
particular child. In any case, before using a deletion type such as 
prepositions and conjunctions, an experiment should be performed to 
determine whether the effectiveness of prepositions and conjunctions 
as found in the present experiment is a function of either the degree or 
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nature of self -reinforcement involved, or whether it is a function 
of the nature of the preposition and oonjunotion. If the former ia 
the oase we may find using reinforced material less effective than 
some other form of deletion* If the latter is the oase it would 
suggest that the filling in of prepositions and conjunctions would be 
superior for at least the initial learning for such a teaching of 
language sequences. 

SUMMARI AND CONCLUSIONS 

A series of twenty-four dose procedure exercises followed by 
twelve comprehension questions was given to 123 ninth grade pupils. 

Eaoh individual was randomly assigned to a particular kind of deletion 
ranging from no deletions as the control group to random deletions , 
noun deletions, verb deletions , modifier deletions, preposition and 
conjunction deletions, and noun determiner deletions. All deletions 
were at the ten percent level. Data for the first exercise, the tenth 
exercise and the twenty-second exercise were presented. The results 
indicate the following: 

1 ) With the exception of modifier deletions and noun determiner 
deletions, individuals tend to make fewer correct closures is the 
process continues* 

2) Individuals tend to make more correct noun closures in the 
tenth exercise. 

3) Individuals who received random deletions, modifier deletions, 
or preposition and conjunction deletions tended to do better at filling 
in correct closures than did the oontrol group with no deletions, on 






the final test. 

4) Where dLose prooedure li used as a part of a sequence of 
tests, ona can expeat a decrease in aoora at a funotion of tha repe- 
tition of oloaa material, 

5) Close prooadura should hava utility as a technique for 
taaohing standard syntaotio pattamo to ohildran of substandard 
language. 



CHAPTER VIH 



THE EFFECT OF NONOVERT REENFORCED CLOZE PROCED0RE 
UPON READING COMPREHENSION 

R&nforoed dost procedure has been used in a four somewhat 
different from the present experiment as a teaching device for reading 
comprehension. The study of Bloomer (1962) shows that where dose 
procedure is reinforced in a motivational scheme, effective increases 
in reading comprehension occur. There appear to be two strands of 
research dealing with the dose procedure. The first, using dose 
procedure as a readability or reading-testing technique, uses non- 
reinf breed dose procedure. The seoond uses dose procedure as a 
teaching device, employs some form of knowledge of results or rein- 
forcement subsequent to the dosures, as we saw in Chapter VI. It can 
be hypothesised, however, that non-reinforced dose procedure has built 
into it a certain dement of reinforcement in that, upon completing a 
closure, the individual is able to reinforce himsdf by matbhing his 
language patterns with those of the author, or rather those he has 
reconstructed from his own and the author's patterns. This would lead 
one to expect that the non-overt reinforoed dose procedure might also 
have involved certain dements of teaching which would tend toward 
improvement as an individual proceeded through a series of dose tests. 
The present study therefore is an attempt to measure the effects of a 
protracted series of non— reinforced dose procedure exercises upon the 
reading comprehension ability. 
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Tiro methods of measuring the affects of non-overt reinforced 
dose procedure upon reading comprehension are available in the present 
experimental design* The first is through the use of the comprehension 
tests subsequent to the dose exercises* If non-overt reinforced dose 
is effective as a method of teaching reading comprehension , one would 
oxpeot an increase in Immediate oomprehension scores or in final com- 
prehension scores, as the series continued* 

The second measurement technique involves the use of standardised 
tests to measure pre- and post-test differences* Here significant 
differences between oontrol and experimental post-tests would attest 
to the efficacy of non-overt reinforced dose as a oomprehension 
teaching tool* 

PROCEDURE 

Pstlor to the onset of the experiment, the Differential Aptitude 
Test had been given to the pupils as a pre-test* The post-test, the 
Iowa Teat of Basic Skills, was administered to ninth grade pupils 
subsequent to the completion of the exercises* The sections administered 
were: the language section, the vooabulaxy section, and the reading 
section* 

P*e- and post-test soores were obtained for 125 ninth grade 
pupils who had each completed at least eighteen dose procedure exercises. 

In addition, each of the dose exercises taken by the students was 
in effect a reading oomprehension test, with the dosen reading compre- 
hension questions which followed it* For present purposes, the number 
of correct reading oomprehension questions for the first, fourth, seventh 
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tenth, fourteenth , sixteenth, nineteenth end twenty— se oond oomprehen- 
sion exercises were also examined* If the non-reinforoed oloze proced- 
ure were to have an effect upon reading comprehension one would expect 
an inorease in correct responses toward the end of the sequence. 

RESULTS 

Table XXIII gives the pre-test* No significant differences were 
found for the DAT verbal soores* One difference approached significance, 
the difference between Type 0 (the control group) and type 5 (prepositions 
and conjunctions) s 1 1 * was equal to 1*97 with 44 degrees of freedom* 
Essentially, then, the randomly selected groups were approximately 
equal in verbal ability as measured by the Differential Aptitude Test 
at the outset of the experiment* 

Table XXIV gives the means and standard deviations for the post- 
test scores from the Iowa Test of Basic Skills. The significant differ- 
ences of the post-test involved Type 4 (modifiers) over the control group, 
***& 5 (prepositions and conjunctions) over the control group for 

the spelling sub-test (L-1 ) of the Iowa Test of Basic Skills? the 
comparison of the modifier deletion group with the control group (t * 3.14) 
and the comparison of the preposition and conjunction deletion group with 
the control group (t * 2.6?). No other significant differences are 
found* However, Type 6 approaches significance over the oontrol group 
for the vocabulary sub-test where t * 1 *97 with 4? degrees of freedom. 

The data do not indicate, therefore, that the use of non-reinforoed 
cloze procedure material produces significant inorease in reading or in 
language skills exoept in spelling, where the deletion of either modifiers 
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or pro position* and conjunctions appears to increase spalling ability. 



Irdlvidual Comprehensi on Taata Following Close BserdlSea. One 
would anticipate that if ona learns through using non-overt reinforced 
doss that the comprehension of passages would become somewhat superior 
with practice. On the other hand, since all of the information for 
comprehension is given to the oontrol group with no deletions, one could 
further anticipate that this group would in turn become superior. No 
significant differences were found. 

Table XXV gives the means and standard deviations for the number 
of oorrect comprehension questions for each type for exercises one, four, 
seven, ten, thirteen, sixteen, nineteen and twenty-two. Neither of the 
above suppositions appears to be the case. 



Table* XXVI gives the significant oritical ratio for comparisons 
between the oontrol group and the experimental groups. Significant dif- 
ferences indicating the superiority of the oontrol group exist for exercise 
ten, where the oontrol group (no deletions) is significantly superior to 
Type 1 (random deletions), lype 2 (noun deletions), and Type 4- (modifier 
deletions). On the other hand, for exeroise thirteen. Type 5 (prepositions 
and oonjunotions) is significantly superior to the control group. Again, 
in exercise twenty-two, the oontrol group (no deletions) is superior to 
Type 1 (random deletions) and Type 6 (noun determiner deletions) , and 
approaches significance (t * 1.96 with 28 degrees of freedom) for Type 3 # 
(verb deletions). These data indicate dearly that the use of non— 
reinforced dose procedure does not inorease reading ability either 
during the process of praotloe with the dose procedure or as a function 
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Mo Delations 27 X 8.19 7.85 7.64 7.68 7.43 7.61 8.10 8.19 

SD 1.96 2.52 2.76 1.80 2.63 2.71 2.00 2.16 
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SIGNIFICANT 't' TEST 0CHPAH1S0N3 OP CORRECT O&iFREHUISION QUESTIONS 
FOR EXPERIMENTAL AND CONTROL GROUP (TIPS 0) BX EXERCISE 



Deletion 

N 1 4 7 10 13 16 19 22 



1# Random 


2.16 


2.40 


2# Nouns 


2.61 


3* Verbs 


1.96 


4. iiodlfiers 

5 . Prepositions and 


2.07 


Conjunctions 


3.17 




6 # Noun Determiners 


2.11 



of the post-test; nor do they substantiate the proposi ,;'.on that the 
greater mount of information present in the no-deletions material 
produces an increase in reading comprehension, though there were 
significant differences, most of which favored the Type 0 (no deletion) 
group. Nonetheless, there were many instances of negative comparison, 
although only one of these was significant,. The present data indioate 
that nan-reinforeed dose procedure material does not affect the reading 
ability of an individual any more or less than standard paragraph com- 
prehension exercises. Further, it is indicated from the mean oorrect 
comprehension questions that neither non-rednforcod doze material nor 

practice with paragraph comprehension materials had any effect upon 
reading at all. 

Correlations of Reading T«»t. Table XXVII gi ves the 

correlation of the reeding test scores with the number of oorrect 
dosures for Exercises 1 , 10. and 22, by ddetlon type. In general the 
correlations are positive, significant at the .01 levd, and in the 
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Table XVII 



CORRELATIONS OP READING TEST SCORES WITH NUMBER OP CORRECT CLOSURES 



FOR EXERCISES 1 , 10 AND 22 BY DELETION TYPE 



Deletion 




Pretest 
DAT VR 




Post-Test 
Iowa Reading 




Tvne 


1 


10 


22 




10 


22 


1 • Random 


•34 


.39 


.65 


.55 


-.07 


.65 


2. Nouns 


.75 


.34 


.45 


•68 


.27 


.58 


3. Verbs 


.86 


.46 


-35 


.26 


.75 


<*> 

00 

• 


4. Modifiers 


.72 


—.06 


.67 


.52 


.07 


•54 


5* Prepositions and 
Conjunctions 


.54 


o 

CM 

. 


•40 


•62 


• 28 


.54 


6. Noun Determiners 
I"""" 1 — 


.33 


.31 


.54 


.60 


.61 


.60 



Minus sign indicates significance in favor of control group., 
r.05 « .19 r.01 * .25 



expected direction. Exceptions to this occur where negative correlations 
are found between . reading test scores on the pre-test and a negative 
correlation for the group doing modifier deletions on the tenth exer- 
cise t and a significant correlation for the group doing verb deletions 
on the twenty-second exercise- The correlations for the tenth exercise 
appeared to be generally somewhat lower than those for either the first 
or the twenty-second. Correlations with the post-test which do not fit 
the pattern oocur for the tenth exeroise t first for the random dolotion 
group and second for tho modifier deletion group. In general it appears 
safe to oondude that the reading ability of n in t h grade pupils is 
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related to thoir ability to coraploto oorroot olosuros. 

Table XVIH gives the correlation of reading tost scores with 
the number of correct comprehension oxeroisos for Exorcises 1 , 10 and 
22 by deletion type. This table gives an indication of greater 



Table mil 

CORRELATION OP READING TEST SCORES WITH NUMBER OP COMPREHENSION 
ITEMS CORRECT FOR EXERCISES 1 , 10 AND 22 BY DELETION TYPE* 



Deletion 




Protest 
DAT VR 




Post-Test 
Iowa Reading 


Tvi>o 


1 


lo 


22 




10 


22 


0, No Deletion 


.34 


.05 


.38 


.39 


.55 


.50 


1, Random 


.01 


.06 


.41 


.23 


r's 

o 

a 

i 


.75 


2, Nouns 


.59 


.86 


.70 


.64 


.60 


.73 


3, Verbs 


.73 


00 

a 

i 


-.15 


.64 


.63 


.79 


4. Modifiers 


-.22 


-3- 

o 

a 
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-.04 


o 

a 

i 


-.23 - 


>•68 


5* Propositions 
and Conjunctions 


.18 


.15 


.39 




-.11 


llii 
a “ “ 


6, Noun Determiners 
, ... . 


.59 


.27 


.19 


.<(8 


.41 


.66 



r.05 ■ .19 



r.01 ® ,25 



variability than that for number of oorroot closures. Horo only tho 
noun deletion group producos all significant correlations in tho 
expected direction, and thoso correlations are generally higher than 
those of tho control group, Tho modifior group produced correlations 
all in tho negative direction indicating that tho offoot of modifier 
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dotations is such that tho comprehension of tho matorlal in question 
is negatively affootod by tho modifier dotation* This way bo soon a* 
a function of, first, tho rolativo difficulty of modiflor dotations; 
and sooond, tho probability that information givon through modifiers 
may bo moro important than that of othor parts of speech in answoring 
oomprohonsion questions* 

VLSCmSHM 

Tho non-ovort reinforced do so proooduro doos not appear to 
affoet tho reading oomprohonsion of students to a different degree 
than doos non-reinforood reading oomprohonsion oxercisos* This suggests 
that tho value of tho dose proooduro exorcise for teaching reading com- 
prehension is a function of tho dose proooduro plus tho method by 
which it has been delivered. We cannot expect tho dose procedure to 
produce gains in reading comprehension under all conditions of presenta- 
tion* There are, however, striking differences between the present 
•tudy and the study of Sehneyer 31 whioh found no significant Increases, 
and the oarlier study of Bloomer 1 and that of Roosniek. 34 Subjects 
In the Sohnoyor study were reinforced by tho toaohor, who returned tho 
corrected dose fill-lna to the etudonte on tho day following the oloee 
proooduro exercise. Scoring wae done in tome of the preolee word only, 
meaning that to achieve a oorreot aoore the student oust fill In the 
exsot word whioh the author Intended. In the earlier Blocmer atudy, the 
studenta graded their papere themselves for the exact word, and then tho 
papere were turned in to be graded by the Instructor for oorreot meaning. 
This difference in procedure may in and of Itself be of elgnlfloanoe for 
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eomprehension gains, in that a pupil did not receive reinforcement 
under the Schneyer technique where a word was perfectly appropriate. 

Hence the student would be likely to generate some considerable frustra- 

tion related to correct language usage. In the Bloomer study any 

frustration generated by using oorreot, out not the author's, language 

would be dispelled at the subsequent meeting by the regrading of the 
instructor. 

In the Schneyer study as in the present study, the students were 
bound to complete all of the exeroises; hence there was no benefit for 
an extremely accurate performance. In the earlier Bloomer study, however, 
pupils advanced through grade levels more rapidly if their responses 
reached a 95% accuracy level. This meant essentially that any frustra- 
tion developed as a function of being involved with the doze procedure 
oould be ameliorated most effectively by good performance. It appears 
probable, therefore, that the doze procedure-methodological combination 
which is effective will indude two dements in addition to doze 
material: first, reinforcement by the teacher for the oorreot response, 
regardless of whether the response is the exact word of the author; and 
second, a motivational scheme with which the pupil's movement through 
the exeroises will be determined by the quality of his response. 

SUMMARY 

The present analysis suggests that the use of non-reinforced doze 
procedure is not effective for producing an increase in reading compre- 
hension. The doze procedure as an effective teaching device therefore 
represents a combination of the use of the material itself in addition 
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to methodology which would appear to contain at least two variables; 
first is reinforcement for a correct response regardless of whether or 
not it is the precise word used by the author. Second, a motivational 
arrangement whereby the individual's progress through the dose pro- 
cedure material is related to the quality of his responses. 

It appears from the correlational data that the ability to make 
dosures is related to the reading ability of the student. However, 
the ability to answer comprehension questions subsequent to a dose 
procedure exercise is dependent upon the type of dose procedure exer- 
cise. dose procedure deletions, such as noun ddetions, apparently 
offer more information and are more like strdght reading material 
than oorrdntione of modifiers. On the other hand, the ddetion of 
modifiers appears to distort the meaning of the passage to a sufficient 
degree as to make the quality of the reading skill of the subject 
irrdevant to the number of comprehension questions answered correctly. 



CHAPTER IX 

rae RELATION or non-reinporced cloze. proceddre 

TO THE INTELLIGENCE OF THE RESPONDENT 

Ho«t reading and verbal activities bear some relationship to 
the intelligence of the individual making responses. One hundred and 
six ninth grade pupils who were Involved in the experiment also took 
as a part of the experiment the California Test of Mental Maturity, 
Soores on the CIMM could then be correlated with the measures of dose 
procedure responding. For the present study the total number of cor- 
rect comprehension questions was correlated with CTMM soores as well 
as the total number of oorreet dosures in the training exercises. 

For sequential comparison purposes, number of correct dosures on the 
first# tenth and twenty-second exercise for each individual was cor- 
rdated with CTMM scores. To test final results, the CTMM was corre- 
lated with the number of correct questions on the recall final, the 
•water of correct reoall questions rdating to exercises No, 1, 10, 

and 22, and the number of oorreet dosures by syntax as well as by 
training deletion type. 

The experimental function of the CTMM was to determine differ- 
ences between the randomly assigned experimental groups. Table T~IX 
glva, the naan language, non-languege, and total IQ moan scores for each 
experimental group and the oontrol group. There are no significant 
differences between the group. In Intelligence. 

Por Interpretation purposes high correlations between in telligence 

89 



o 



Table XXIX 

CALIFORNIA TEST OF MENTAL MATURITY 
MEAN SCORES ST DELETION TIPS 



Delation Typt 


Statist! 0 


!S 

Language 


V7 

Non»Languag# 


- IS 
Total 


0. No Dilation 


X 


109.53 


118.74 


114.32 




SD 


17.36 


16.23 


14.40 


1 • Random 


X 


114.60 


119.22 


117.22 




SD 


14.52 


17.11 


12.77 


2. Noun 


X 


111.71 


122.71 


116.00 




SD 


17.03 


22.72 


16.85 


3» Varb 


X 


103.00 


111.50 


107.30 




SD 


14.08 


27.50 


20.41 


4. ModL£Ur 


X 


114.92 


129.42 


122.41 




SD 


13.59 


17.11 


11.58 


5« Proposition and 
Conjunction 


X 

SD 


115.56 

9.80 


121.44 

16.10 


118.94 

10.99 


6» Noun Detezminer 


X 


109.07 


119.20 


114.40 




SD 


15.82 


13.06 


12.63 



Table XXX 

MEAN CORRELATIONS FOR CAUPOSNIA TEST OF MENTAL MATURITY SCORES 
WITH NUMBER OF COMPREHENSION QUESTIONS CORRECT 
BY TYPE OF SYNTACTIC DELETION 



CTMK Sooro 



Tr ^ nl " g lwl 'i1flTn. Tnrrr 

2 4 




. 26 ** . 30 ** . 83 ** 

Non-Unguage .24** . 22 * . 77 ** 

Totu .‘I 0 ** • 26 ** .82** 

i* 3 ™* *°? u ** 1 oonfldeno* 
♦•Slgnifloant beyond .01 level of «r».^de nce 



.25* 

. 34*0 

. 49 ** 

1 



.27** 
. 0 ? 
•08 
1 



. 26 ** 

.29** 

.32** 

19 
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«d radon* olote scores Indloato the subject, beoause of the nature 
of the delation, uses hie intelligence fox* solutions to the problems 
presented in the materials. Low correlations indicate the nature of 
the material is suoh as to preolude the use of Intelligence for 
solution. Suoh a ease might ooour when the material is completely 
unintelligible or where individuals of lew intelligence can do as well 
as those of high intelligence, as in extremely easy material or materi- 
al where the language patterns are firmly established in all individuals. 

Intelligence and Oompreh«nei m . Table XXX gives the correla- 
tions for the California Test of Mental Maturity scores with the number 
of comprehension questions answered correctly by syntax deletion. The 
majority of the correlations are significant. However, oertain varia- 
tions oan be seen between materials of differing deletion types. It 
should be noted that the small number of oases for Type 3 deletion s 
taking the CBM precluded the use of this particular deletion type in 
this comparison, Bxe data Indicate that the deletion type appears to be 
a variable in the relationship between comprehension of a passage and 
intelligence. Type 2 (noun deletions) correlates very strongly, indi- 
cating that when noun, are deleted from a passage the understanding of 
the passage is largely dependent upon the intelligence of the indiv idual 
Those individuals who are more intelligent will tend to understand it 
better, those who are less intelligent will not. On the other hand, if 
we consider Type 0 as a base line in the relationship between intelli- 
gence and reading comprehension, random deletions or the deletions of 
noun determiners do not significantly affect the relationship between 
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reading comprehension and intelligence. This would seem to indicate 
that the disruption in meaning here is relatively insignificant and 
for ninth grade pupils random deletions at the ten percent level or 
deletions of noun determiners do not have a significant impact upon 
the intelligibility or the comprehension of the paragraph for the full 
range of individuals. 

However, when we examine type 5 deletions (prepositions and 
conjunctions) we observe a somewhat different phenomenon. Here the 
correlations all tend to be low, two of them are not significant. This 
would suggest that another variable ie operating such that intelligence 
no longer enters into the process of comprehension through these materials. 

One hypothesis might be that the materials have been rendered 
unintelligible to any degree of intelligence. However, since the degree 
of comprehension questions answered by individuals who have been trained 
with Type 5 deletions (prepositions and conjunctions) is markedly higher 
than most of the other situations, and since this is a function word and 

contributes relatively little to the meaning, this hypothesis is not 
tenable. 

One might also hypothesise that the deletion of a preposition or 
conjunction might involve a certain interruptive px'ooess for the more 
intelligent individuals, but on the other hand would not affect the com- 
prehension of the less intelligent individual, perhaps through his lack 
of recognition that something had been left out. Here again the number 
of correct comprehension questions for Type 5 is relatively higher than 
for other groups, indicating there is relatively little disruption on 
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the part of the more intelligent individuals* 

Evidenoe points to the foot that the deletions of prepositions 
and conjunctions tends to make the material somewhat easier to com- 
prehend and it may be that material which oontains prepositions and 
conjunctions , as the Yoakum Readability Formula would suggest, may 
tend to inhibit the reading of individuals of lower IQ and ostensibly 
also lower reading ability* The elimination of these therefore might be 
seen to serve as clarification for the individuals, the subscript line 
acting in many senses as a rather pronounced punctuation mark* 

Further, in ea q d a i n i n g this phenomenon we must also consider the 
fact of the focusing effect of the close procedure* It might appear as 
a function of the present data that where the doze procedure forces 
the individual to focus an material, but at the same time does not 
offer him a challenge in terms of extreme loss of meaning, that this 
would be beneficial to individuals of lower IQ and tend to make their 
performance somewhat more consistent with that of individuals jf the 
upper IQ, who may be seen as already approaching the upper limits all owed 
by the material* 2hus where the deletion is prepositions and conjunctions 
any relationship between the comprehension of the material and the Intel- 
ligenoe of the individual reader tends to be irrelevant. 

Intelligence and th e Sequence of Exercises. She oorrdation 
between CM! scores and the number of correct comprehension questions 
for three of the sequence of twenty-four exercises was selected. These 
are Exercises No* 1, No* 10, and No* 22* As has been explained above 
in ftfgoyiure. . Exercise No* 1 is not a single exercise but may have 
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been any one of the twenty-four. It ie simply the first exercise 
which was done by the individual. Similarly, exercise No. 10 is the 
tenth in the sequence, and exercise No. 22 the twenty-second in the 
sequence, although they may have been any of the twenty-four exercises 
presented. Table XXXI gives the correlations between the CIMM and 
the number of oorrect comprehension questions for the first, tenth 
and twenty-second exercises. Here no particular trend is noted, and 
the data for all of the deletion types is combined. All of the correla- 
tion coefficients are significant. Those for the language sections of 
the test are approximately equal to those for the non-language sections 
of the test, as well as for those of the total of the test. These data 
indicate that we are getting no gross over-all sequential effect of the 
sustained non-reinforeed close procedure and intelligence. One would 
surmise therefore that consistent use of non-reinforced close procedure 
materials with a group of individuals does not disproportionately 
affect any subjects at different levels of intelligence. 

Table XXH 

CORRELATIONS OP CALIFORNIA TEST OF MENTAL MATURITY 
WITH NUMBER CP OORRECT COMPREHENSION QUESTIONS 
FOR EXERCISES 1 , 10 AND 22 



OTMM Score 


i — 


Exercise 

fo 


22 


Language 


.**5 


.23 


.38 


Non-Language 


.25 


.30 


M 


Total 


.38 


.25 


.60 
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Intelligence and Recall of Comprehenstonlt ems . One would 
typically expect some relationship between recall of items and the 
intelligence of the individual learner. For the present study the 
recall final was given one and one-half weeks subsequent to the 
conclusion of the exercise section of the experiment. The recall test 
consisted of six of the twelve comprehension questions for each of the 
twenty-four exercises taken by an individual. 

The correlations in Table Trim indicate in general a moderately 
significant relationship exists between the intelligence of an individ- 
ual as measured by the CXMtl and the recall of items at periods from 
one and one-half to thirteen and one-half weeks later% Again these 
correlations seem to be most strong for individuals who had been trained 
on Type 2 (noun deletions). The level of relationship for those indi- 
viduals trained on control Type 0 (no deletions) was approximately the 
same as the original correlations. Other correlations tended to be 
considerably lower, however. Two interesting differences did occur. 

The recall of items for Type 6 (noun detenniners) tended to be less 
closely related to intelligence than were the immediate recall compre- 
hension questions. 

The correlations for delayed recall tend to be somewhat lower 
than those relating immediate recall and intelligence and suggest the 

possibility that reoall is not as closely related to intelligence as 

* 

immediate performance. 

Relations of the CTT4H with the number of oorrect comprehension 
questions on the recall final for the first, tenths and twenty-second 
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Table £0X1 

MEAN CORRELATION OF CALIFORNIA TEST OF MENTAL MATURITY WITH 
SOMBER OF COMPREHENSION QUESTIONS CORRECT ON RECALL FINAL 

BI TIRE OF SXNTACnc DELETION 



CCTM Score 




l 

i 

% 

3 






0 


1 


2 


4 


5 


T 


Language 


.24* 


.25* 


.43** 


-.08 


.34** 


.18 


Non-Language 


* 

CM 

• 


.09 


.31** 


.32** 


.13 


.13 


Total 


.30** 


.23* 


.42** 


♦23* 


« 

CM 

. 


.19 



Table XXXIII 



CORRELATIONS OF CAUFORNIA TEST OF MENTAL MATURITY 
WITH NUMBER OF CORRECT COMPREHENSION QUESTIONS 
ON RECALL FINAL FOR EXERCISES 1 , 10 AND 22 



CBii Score 


i — 


fiserdaes 

10 


22 


Language 


.47** 


.21* 


.24* 


Non-Language 


.03 


.39* 


-.09 


Total 


.20* 


•41 ** 


.13 



exercises taken by the individual tend to support the notion, the 

reduoed dependence of delayed recall and intelligence. More than 

half the correlations for the reoall for exercises 1, 10 and 22 are 

not significantly related to the intelligence of the individual. This 

lack of significance appears to indicate that, while intelligence may 

Play some part in the delayed reoall of items, this relationship is 
not particularly reliable. 
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Thus we find that the comprehension of items with varying 
syntactic deletions indicates a relatively high relationship between 
comprehension and recall of materials with noun deletions. The rela- 
tionship is lower when function words are deleted, particularly for 
the immediate recall of material delivered in a context of missing 
prepositions and conjunctions. Further, during a long series of close 
materials, there appears to be no change in the relationship between 
intelligence **nd the ability to answer comprehension questions. 

The delayed recall of items relating to previous exercises, 
with the exception of materials of noun deletions, does not indioate 
as strong a relationship between intelligence and the number of correct 
recall items as was found for the number of immediate recall items, 
suggesting that the relationship between intelligence and delayed 
recall is somewhat less than the relationship between intelligence and 
reading comprehension* 

The Relation of In telligence to Correct Closures. Table XXXIV 
indicates that the correlations between CTMM scores and the number of 
correct closures in training exercises is much the same as with com- 
prehension questions. The correlation between correct random or noun 

closures and intelligence is relatively higher than for other closure- 

\\ 

types and the correlation with the number of modifier or preposition 
and conjunction closures is relatively lower* 

Intell^g^poe^^yj CQ raeflt _&osurea on the Final Cloze Teat bv 
S Eypfl! fef^Ty^ln^qg jfelp tskSIl* 2h® final dose test is a random dose 



test. It is interesting to note that individuals who were trained 
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Table XXXIV 

MEAN CORRELATIONS FOR CALIFORNIA TEST OF MENTAL MATURITY 
WITH NIMBER OP CORRECT CLOSURES IN TRAINING EXERCISES 

BY TYPE QF SYNTAX 



CTMM Score 


— gaining 


Deletion Tvdo 






0 1 


2 


% 


5 


'S_ 


Language 


.1*8** 


• 2ft** 


00 

o 

e 


.38* 


.06 


Non-Language 


.31 ** 


•ft2* 


.20* 


.11 


.56* 


Total 


.31** 


. 60 * 


.21* 


.25* 


.ftl* 



xaiuLe jlaav 



MEAN CORRELATION FOR CALIFORNIA TEST OF MENTAL MATURITY 
WITH TOTAL CORRECT CLOSURES ON THE FINAL CLOZE TEST 
BY TYPE OF SYNTACTIC DELETION 



CTMM Soore 



Language 

Non-Language 

Total 




.3/** 


.09 


.26** 


o 

« 

i 


.36** 


• 2ft* 


.19 


.13 


.25* 


.38** 


.25* 


•iy 


.3^* 


.17 


. , *0** 


.2?** 


.33** 


.25* 
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with random dose tests have extremely low correlations between 
intelligence and the number of oorreot responses made in the final 
dose test (see Table XXXV). Hiring training, however, this relation- 
ship was not in evidence as significant correlations wore obtained 
between training in random closures and intelligence. Hie indication 
here is that this is a training effect, in that these individuals are 
being tested in the mode in whioh they were trained and hence as the 
training has be?n effective the requirement for intelligent behavior 
is somewhat lessened. Individuals who were trained on specific syntax 
deletions or no deletions at all generally tended to indicate moderate 

significant correlation between intelligence and the ability to perform 
on the final close tost. 

Intelligence and the Num b er of Correct Closures for +he First . 
£e pth and flwepty-Socond flxer d sea j n the Training Session- If the 
hypothesis that exposure to non-reinforced dose procedure materials 
in faot trains an individual to perform dosures, one would expect 
a certain dimunition of correlation coefficients between intelligence 
and the number of correct dcsure* between the first and the twenty- 
second training session. This is evident from the data presented in 
fable XXXVI. The differences, however, are not significant, but since 
here we are dealing not only with individuals trained via the random 
dose material but also individuals trained by other syntactic deletions, 
one would expect any differences to be ocduded by this particular faot. 
This conjeoture is uphdd by the correlations for Type 1 (r&ndom dele- 
tions) and the language section of the CTMM. For the first exercise 
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the correlation is .51 1 for the tenth exerolse it ie .45; and for the * 
twenty-seoond it is .33» showing a gradual diminution in the relation- 
ship. The .33 with twenty-six degrees of freedom is not significant 
at the .05 level. 

The proposition that individuals trained on random dose proced- 
ure learn to make dosures more effectively and hence are not required 
to use as muoh intelligence on the final dose test ae individuals 
trained through other procedure is not substantiated when one looks at 
non-Xaxv£uage intelligence, or a total which reflects both language and 
non-l&tiguage. The rdationship between non-language intelligence and 
the number of correct dosures moves from -.20 in the initial exercise 
to .30 in the final exercise. Neither of these are significant. How- 
ever, the contributory effect of non-language intelligence shifts the 
total scores from .24 for the relationship between the CTMM total and 
the number of correot dosures on the first exercise to a correlation of 
• 50. On the twenty-second, this last is of course significant at the .01 
level of oonfidenoe. This particular trend for the non-language and 
the total CTMM scores is not followed by the combined total groups. 

Table XXXVI 

CORRELATIONS OF CALIFORNIA TEST OF MENTAL MATURITY 
WITH NUMBER OF CORRECT CLOSURES 



FOR EKRCTSES 1, 10 AND 22 


CXMM-Soore 




Sgerdeea 




1 


10 


22 


Language 

Non-Language 

Total 


.40** 

.38** 

.5?** 


.20* 

.42** 

.39** 


.31** 

.21* 

.39** 
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ca.g»e Teat by Syntaotio Tvi hi. This comparison indicates ths effects 
of the syntax of the deletion on the final olose test itself, regardless 
of the syntax of the training procedure. In general, significant 
correlations are found between intelligence and the number of meaning- 
oarrying words correctly fill in (see Table XXXVII). However, when we 
consider function words, particularly prepositions and conjunction, the 
relationship does not hold. This would suggest that function words 
or grammatical structure are basically learned by every individual 
and hence do not represent an intellectual type of problem. 

Table XXXVII 

CORRELATIONS OP CALIFORNIA TEST OP MBJTAL MATURITY 
WITH NUMBER OP CLOSURES BY SYNTACTIC TYPE 
ON CLOZE FINAL TEST 



CUM SCORE 




_Tvoe of Deletion 






2 


3 


5 


5 


6 


Language 


.41** 


.39** 


.20 


.15 


.10 


Non-Language 


.37** 


.15 


.26** , 


,11 


.27** 


Total 


.47** 


•40** 


.2?** 


,01 


.25* 



*.05 * .20 
r.01 * .26 



SUMMARY 

The CTMM was given to 106 of the population in the experiment. 
No significant differences between the groups were found. In general 
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correlations of dost toorts and intelligence were positive and 
significant* However, in dealing with extremely difficult ox extremely 
simple closures the correlations were low and in some oases negative* 
This phenomenon with the more difficult materials suggests that the 

deletions tend to decrease intelligibility of the selection to the 

* 

point that intelligent behavior represents no advantage* Where the 
low correlations occurred with extremely simple materials, it appeals 
that the language patterns are either well learned by all Individuals 

or the deletions have the effect of shortening sentences to reduoe 
readability* 
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. CHAPTER X 

4 

AREAS FOR FUTURE EXPLORATION 0# CLOZE PROCEDURE 

Study of the dose procedure and its utility is relatively new 
and henoe has been confined to the areas of readability, comprehension 
testing and reading comprehension improvement,, The study by Hafner 
has suggested a number of other potential uses for doze. The 
present chapter proposes to suggest several additional uses or improve- 
ments of present uses which might be fruitful areas for exploration. 

doze Readabilit y and Comprehension Testing. The value of the 
doze procedure as a readability predictor has been well established 
by Taylor. Reviews of doze research such as that of Rankin 
show an increasing range of possibilities for experiment in doze 
procedure readability. The present doze techniques allow for putting 
materials in rank order for any particular group of individuals. It 
does seem surprising that some attempt has not been made to generate 
a standardized scale so that readability by the doze procedure might 
be converted to some useful term such as grade level. Part of the 
reason may be that grade level is perhaps most useful at th very early 
grades and we find that doze procedure is of relatively little use at 
early grade levels* Present first grade materials with many repetitions 
is much more difficult than second grade material, at least for adults. 
However, even in terns of grading material at the higher grade levds 
one must first of all establish a standard for the doze procedure 
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itself. The standard must be in terns of a peroent of deletions as 
well as the types of deletions. One must also make the decision as 
to whether to be concerned with the precise word written by the author 
or with any correct meaning the respondent may supply. As Taylor 
suggests , these two scores correlate extremely well with each other. 
However, certain applications are suited to generalized meanings 
whereas others, such as the measurement of readability, may be more 

suited to specific meanings in that the scoring becomes considerably 
easier. 

The scaling system in turn would be somewhat complex in that 
it also necessarily should take into aooount the reading level of the 
individuals used to grade the material, or use large sample techniques. 

One problem arises with the use of doze procedure in the class* 
room as a readability measure. If one takes a present dass to estab- 
lish the readability of material, we know only ex post faoto that the 
material is either too difficult or too easy. These problems notwith- 
standing, a scaling system would be of great value because of the high 
reliability with which doze can be scored, and the relative flexibility 
of doze to adapt to the evaluation of many kinds of situations. 

As with the application of any readability formula, more experi- 
mentation is required to determine the proper reading-teaching level. 

The question remains t "Shall we use relatively easier material and make 
the child fed comfortable in reading, or shall we use relatively more 
difficult material to make the child feel challenged by what he is 
doing. Probabilities are that different applications apply to 
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different children or different types of children* The appropriate 
rules for this kind of application are still missing* There clearly 
must be some systematic relationship between the child, the situation 
in which he is reading, the nature of the material he is reading, and 

the reading level or difficulty of the material, and his teacher. 

% 

doze P rocedure Comprehension Testing. The seoond area well 
established in oloze research is that of the testing of reading compre- 
hension* doze provides reliable mechanism for testing comprehension, 
and here again there is a requirement for standardization* Cloze as 
a testing device can be readily administered and scored by teachers 
at a moment 3 s notioe, and yet to be in practical terms for a teacher 
a series of paragraphs, scaled for the number of errors by children 
of varying reading capacities, would be In order. Standardized decis- 
ions relating to type of syntactical deletion or percent of deletion 
would be a function of the author per se, but oer tainiy such a device 
should prove extremely useful* 

gloze Procedure as an I ntelligence Test* The study has shown 
a relationship between the number of correct closures and the intel- 
ligence of the responder. This relationship has been particularly 
strong where nouns were deleted from the material* Given the premise 
that intelligence is related to verbal ability, and verbal ability in 
turn related to 'the frequence of the vocabulary an individual uses, 
the correction factor for the frequency of occurrence of the word 
substituted might well serve to increase the correlation and hence 
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the efficiency of the oloze as a measure of intelligence* Such a test 
would be related in part to an individual's ability to read* In 
addition, such a test would not necessarily be culture*free* The 
scores would depend, as would all doze procedure responses, upon the 
language patterns of the individual* On the other hand, the test would 
be simple to administer, and would in turn be easily corrected for the 
individual's reading level* It would give an additional bonus to the 
individual who tends to use language in a creative fashion as the 
tendency to use cliches would mitigate against high scores* The problem 
with using oloze, as with nearly any other intelligence test, is that it 
would also be possible to train an individual to achieve a better score* 

The use of the doze procedure as a device to measure intelligence 
is predicated on the fact that one should be able to increase the rela- 
tionship between other measures of intelligence and the doze procedure 
by weighting responses according to their frequency of occurrence* It 
would appear similarly that doze procedure with the deletion of function 
words such as prepositions and conjunctions, or noun determiners, would 
probably be ineffective* 

Q-.S89 Procedure as an Aptitude Based upon the premise 

that an individual's language and ability to handle concepts is superior 
in an area of interest and potential specific aptitude tests using doze 
procedure oould be devised, these aptitude tests would afford, as did 
the intelligence test, a weighted score for infrequent or spedalty 
bound words. Materials of varying levels in the social sciences, the 
sciences or any vooational area, can be graded in difficulty to give an 
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indication of an Individuals ability to handle conceptualizations 
in any particular field. A well-developed battery of these materials 
with several levels of materials in important areas, with corrections 



for the individual's age and reading ability, might well afford an 
effective predictor of speoifio academic or vocational suocess. 



as a Tei 



.o Ability. While individual 



syntax is no longer considered a serious problem, where an individual 



syntax is to be converted to standard syntax it would seem that the 
doze procedure could be used a, a test of standard syntactic ability, 

’n that oertain syntaotio problems could be incorporated in the blanks 
of one or several doze procedure passages. Reading level and syntactic 



ability would tend to interact in suoh an instrument. 



However, this 



interaction could be largely avoided, except for cases of extreme 
reading disability, by making the doze material selected of extremely 
low readability. Since doze is of little value below fourth or fi th 



grade level because of the language pattern development of children, 
material whose initial reading level was of second or third grade level 
before doze mutilation should present rdatively few reading problems 
except to the most seriously disabled readers, dearly, as the level 
of the child to be tested increases the readability of the material may 
also be increased but in any case should remain several grade levds 
below the mean for the children to be tested. 

It would be possible to generate a wider range test from a single 
selection, ^tactic ability under these circumstances would not be 
independent of an individual's reading ability and therefore sub-scores 
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relating to one’s ability to fill in words of particular syntax or to 
solve particular syntactical problems would be related to the reading 
ability of the child* For instance 9 the child at fourth grade reading 
level who is able to solve seven out of twenty auxiliary verb problems 
flight be operating at normal level 9 whereas the child at seventh grade 
level might be expected to resolve correctly fifteen of the twenty 
auxiliary verb problems* Scoring would necessarily have to be in terras 
of the syntax of the supplied word as well as in terras of the meaning 
of the supplied word* The syntactic problems would have to be chosen 
with great oare to fit most probable errors; however, it would appear 
tlaat this kind of test would afford the testor the possibility of draw* 
ing both reading level and syntactical level from the same instrument 
by using a context which, for an individual, is relatively more normal. 

The present experiment has indicated some potential problems 
which may result in lowered reliability of cloze materials either as 
a readability testing device or as a reading comprehension device 
where different passages are used to measure the same group of children. 
The first circumstance occurs where the style of the author is rela- 
tively even, such that the sentence lengths are approximately the same, 
and the structure of the sentences is also relatively the same* Where 
evory nth word, the deleted word, represents the length of the sentence, 
then each deletion will be of the same syntax, and a distorted result 
would be obtained. The second oireumstanoe is similar, but should by 
chance an inordinately large block of deletions be of the same syntax 
or syntaxes of similar difficulty, one might also get a disproportionate 
result. It is dear that the danger of this difficulty increases as 
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the proportion of closures per sample size decreases. Hence in a 
large number of situations, one deletion in five words would tend to 
be somewhat more reliable than one deletion in ten words, in turn 
more reliable than one deletion in twenty words, etc. Users of doze 
material must guard against this problem. The relative proportion of 
deletions of any particular syntax should approximate the proportion 
of the occurrences of that syntax in the material. Such a check should 
be made on any random doze material, A similar check is not necessary 
with syntactic deletions as long as appropriate compensations are made 
for the relative difficulty of the syntax deleted. 

The Cloze Procedure as a Controlled Protective Device , Tiae 
doze procedure as it stands bears a relationship to sentence- completion 
type personality tests. Cloze procedure responses appear to vary with 
the personality of the individual filling in the dosures, and seem to 
form a consistent pattern. Sentence completion techniques br«ng up a 
consistently varied sequence of problem situations through the cue 
stem. The doze procedure, however, oould be used to measure the con- 
sistency of a particular reaction by giving a longer contextual associ- 
ation relating to a particular kind of problem. The doze context 
would determine the nature of the response to some degree by determining 
the theme of the problem. Problems would be built in such a way that an 
individual may make a number of possible answers depending upon his basic 
hostility, insecurity or security, or sociability, or other dominant 
personality theme. The examiner would be concerned not only with the 
nature of the response but also with the consistence of the response to 
a particular doze passage or through several different doze passages, 
o 
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This would serve to give us a much more reliable estimate than is 

presently found in sentence completion type materials. The following 
represent examples of materials* 

■ — >■ .. did not like h i m and others thou gh t 

wa ® — When he went to _______ he was 

always 



Sometimes 



would ask hlrn 



to 



or : 



Suzan was a little girl. Her mother and 

father often asked her to , h, r mother and 

father were never 

— dnd sometimes she 



,• She liked to play best 



Further, it would be possible to develop an objective scor in g 
device using the cloze procedure. One method of developing an objective 
scoring system would be to use the semantic differential to estimate the 
position of various words on a continuum. Each word substituted in the 
doze passage might therefore be graded on a number of dimensions. The 
relative sum of words for eaeh of the dimensions, the sum of the soores 
for any particular dimension, would give the propensity of an individual 
to move into or to be affected by that particular dimension. Standard 
scores could be developed for Individuals of varying age levels. 

The selection of the type of response to be omitted be 

fairly oarefully done in using the doze as projeotive. Hints as to 
the form or the particular thing desired should bo deleted. Hence if a 
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single noun is deleted and it reoccurs it should be deleted each time. 
It would be most common to delete verbs or modifiers as these tend to 
convey the greatest amount of emotion. The deletion of structure words 
would allow the reader to determine the direction that the material 
would take. 

For the purpose of the projective cloze, the material must be 
of emotional significance, dealing with a swlected number of problems 
which represent significant dimensions of personality. The material 
should be designed such that stories can evolve in different ways if 
different words are selected. Thus the character of the story will be 
a function of the individual producing the story. 

* * 

Cloze as a Teaching Instrument. The doze procedure at this 
juncture appears to have four possibilities as a teaching instrument. 

In each case the doze procedure would be handled somewhat differently 
to produce desired results. The four possibilities are: the teaching 

of grammar syntax; the teaching of composition; the increase of read- 
ing comprehension; and the teaohing of content. 

Grammar and Syntax. The data in the present study with non- 
reinforoed doze procedure indicates an increase in the ability to 
make dosures of certain specified syntaxes of the syntactic categories 
used. Increases were noted in the ability to fill in prepositions and 
conjunctions, noun determiners, modifiers, and in same instances, verbs. 
Random doze procedure training of a non-reinforoed variety also pro- 
duced an increase in the ability to fill in words of several syntaxes. 

It appears as though the non— reinforced doze procedure, however, is 
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inef f p otive iith nouns , and relatively ineffective with verba* 

It should be noted that doze procedure exercises bear a 
certain similarity to the sentence type exercise which is customarily 
used for the teaching of grammatical structure in workbook-type 
materials. Workbook materials typically consist of a sentence in 
which a blank has been left for the individual to fill in, involving 
some syntactic problem* Sometimes the blanks are presented with 
multiple choices and sometimes with constructed responses* The 
probability of a correct response is controlled by the selection of 
the technique of presenting the correct answer* Further, these 
materials are typically reinforced by an answer key at the back, or 
by teacher or student correction of the material subsequent to the 
working of a page* These materials have the disadvantage that they 
are discontinuous sentences while normal speech or writing is more 
continuous in form* This objection oould be avoided through the use 
of the cloze procedure* 

Use of the doze procedure has not only increased the number of 
dosures of the appropriate syntaxes but also the number of specific 
words the author used* This suggests that the doze procedure tends 
to produce both more conventional language patterns and a higher degree 
of syntactic accuracy* 

It should be also noted that the use of non-reinforced dose 
procedure is relativdy inefficient* It appears necessary to attempt 
an experiment using doze procedure for the teaching of syntax using 
a reinforced form* Theoretically the use of reinforcement would be 
more effective* 
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The use of the dose procedure as a technique for teaching 
syntax requires diagnostic technique,. Within any particular group 
many syntactic errors would be subcultural in that they would be held 
in common by a particular group of students* It is essential to 
determine beforehand which errors are common for a particular child 
and which are common to a group of children. A teacher might devise 
a test again using dose procedure where the teacher spedfically 
deletes words which relate to a syspeoted syntactic error. Needless 
to say there should necessarily be several instances of any suspected 
syntactic error included in the testing materials. The deletions 
need not necessarily be limited to a single word. Phrases of two or 
three words where these involve certain syntactic and graramatic 
problems might be deleted. It should be cautioned that multiple word 
deletions be limited to upper grade students who have sone experience 
with the language and are able to substitute longer phrases with some 
degree of accuracy. Once the test has been used to detenoine the 
problems of a syntactic nature for any particular child or group, 
cloze procedure materials can be developed to correct a particular 
set of substandard syntaxes. Eighteen to twenty such doze exercises 
of about fifty responses each concentrating on specific syntactic 
problems, reinforced by a list of correct responses and also reinforced 
by subsequent checking for correct meanings on the part of the teacher, 
would be an effective method of teaching standard syntax to a child. 

The correct answers for the child would be any words which are meaning* 
ful which also involve the appropriate syntax, or in turn words which 
would adjust or change the syntax in an acceptable manner. An acceptable 
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answer must be In keeping with the total paragraph and not simply 
restricted to the particular sentence with the blank in it. One real 
advantage of this use of dose procedure is that the child must observe 
not only the syntax within the specific sentence but also the context 
in which the sentence occurs. 

Such a procedure would essentially involve experimentation with 
the dose form and other potential techniques for teaching standard 
syntax before widespread adoption of such a technique would be advis- 
able* 



Composition , The teaching of composition is, from the present 
point of view, almost an opposite kind of problem from the teaching of 
syntax. Syntax involves the use of regular and relatively convention- 
alized forms, whereas the goal in composition is the use of unusual 
and inventive ways of expressing oneself. Again, contrary to the 
problem of syntax, th> syntactic forms which carry the greatest weight 
in the composition are such words as verbs, adjectives and nouns. The 
variety and unusualness of these words tend to be related to the polity 
of the composition. 

The Ehglish teacher who is responsible for the teaching of 
composition is beset with the problem that his students* work does 
not match the standards which he desires. A large bulk of the liter- 
ature which relates to composition expresses various techniques for 
increasing the quality of student work. The fact that this kind of 
literature is continuing is testimony to the relatively limited success 
of the techniques offered. The difficulties which the students appear 
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to have lie in three general areas* The first is organization of 
materials* The students appear not to be able to get a generalized 
over-view of the material and hence the material they write is poorly 
organized* Second* *hglish teachers bemoan the number of cliches, the 
paucity of the language of the students* The written expression of 
pupils appears to take a very conventional form and hence does not 
oaptivate the audience nor express in subtlety the limi ts of the 
subject* The third area for concern is the relatively poor syntax of 
the pupils* Some pupils seem to make systematic sets of grammatical 
errors and these appear to be very difficult to correct* 

We can see a relationship between the teaching of syntax and 
the over— use of cliches* The pupil tends to learn to use his language 
in patterns and hence does not weigh each specific word which he utters 
or writes, but rather begins on a particular language chain which will 
lead him to his desired end point and simply fills in from his learned 
language pattern and does not think about choosing the words in between* 
We sail see that many of the syntax and grammar teaching exercises we 
give to children do exactly this— that is to say, they teach the child 
an appropriate language pattern wKch is the same as a oliohe, the 
difference being that for present purposes the syntactical lang u age 
pattern is an acceptable and common convention whereas the cliche 
because of its over-use becomes qualitatively unacceptable. In essence, 
then, the language teacher in some measure works against himself by 
teaching grammatical form as a language pattern, in the same way that 
the teaching of idioms in a foreign language reduces the degree of 
freedom of the child in terms of choice for an expression of a particular 
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concept. If we look at the complaint of the language teacher, that 
the child tends to use few words and to use cliches, we see that this 
complaint resolves itself to the fact that the words used by the child 
are in common patterns and are also words which are highly frequent and 
partly as a function of this, also low in emotional impact. If we can 
judge from the closures in the present experiment, the problem of 
learning a syntax is relatively minute and hence pupils 1 composition 
could be upgraded considerably by increasing the richness of their 
language. The solution therefore to part of the problem of composition 
is to design materials which will increase the variability and breadth 
of the child’s language usage. Here again if we are to judge from the 
vocabulary studies, the number of words with which a child is fami liar 
and can identify is quite large. The problem resolves itself to the 
fact he does not use these words. 

Studies with word association techniques indicate that when an 
individual is confronted with words which have an emotional impact for 
him, a certain group of people, depending upon their particular person- 
ality, will tend to respond in kind; that is to say, if given a highly 
emotional stimulus, most people will respond with a hi ghly emotional 
response; given a relatively neutral stimulus, most people will respond 
with a highly neutral response. If we can therefore develop materials 
which are compositional in nature which give emotional stimuli, we can 
expect an increase in the amount of emotional response we will get and 
theoretically if we give an individual infrequent words we will by the 
same token tend to get less frequent words in return. 
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It is clear that the number of external stimuli in a composition 
situation is relatively limited, usually merely a theme or topio is 
given to the individual and he in turn will respond to this by pro- 
ducing a relatively long chain of responses on or related to this 
particular theme. This presents a problem in composition in that the 
individual is allowed to move from the theme and the stimulation 
produced by it through his own series of response chains, which in 
turn will stimulate him. to move toward his own medium and hence his 

compositions will have relatively smaller degree of variability in 
them. 

Given the fact that an individual responds to verbal stimuli 
by responses in kind, and since we would wish to continue the production 
of highly emotional and infrequent responses on the part of the indi- 
vidual, in order to train the individual to more complex language 
patterns we will have to present not just a theme as a set of ‘ stimuli 
but also a series of continuous stimuli throughout the composition 
which will reinstigate the writer to more emotional and less frequent 
language. The obvious detriment to this kind of system, however, is 
that the individual will then become dependent upon successive sets of 
stimuli throughout his composition and will not be trained to produce 
his own material. However, as we withdraw the amount of stimulation 
in a verbal word association situation we find that the quality of 
responses remains essentially the same, although the variability or 
predictability of responses decreases. Thus, after we have trained an 
individual to respond to emotional stimuli by producing emotional words 
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and respond to infrequent stimuli by producing infrequent words, we 
oan gradually withdraw by a technique of fading similar to that used 
by Skinner, and make the individual more independent and more able to 
generate his own long sequence patterns, 

-f 1110 close procedure offers a technique for accomplishing just 

this end. Typically in the doze procedure an individual draws from 
his own stock of words and his own language patterns to fill in the 
blanks whioh are left in the material. Typically he tends as well, to 
matoh the style of the author, which aacounts for the large number of 
correot responses which children in junior high and high school will 
make to doze procedure material. We see therefore that the cloze 
procedure offers a technique for shaping the responses of the individual. 
9y giving the pupil doze procedure materld in whioh tae s timuli or 
the doze cue, (that is, the word immediately preceding the blank which 
is to be filled in) is a highly stimulating word, we will be likdy to 
have the individual draw upon his own background of highly stimulating 
words to fill in the blank. If we train the individual at one level 
ddetion to fill in words, and then we gradually deorease the amount 
of stimulation and increase the amount of independence whioh the 
individual has, we find that we oan fade the doze procedure from the 
composition situation and successfully train he individual to 
compositions of more highly varied material. Further, the fading 
situation will also be of some assistance in teaching the child to 
organize material, because the first material he gets will be highly 
structured and organized. As he continues to respond, he will have 
to inse t more of his own structure and organization. Hence in some 
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measure the use of the fading doze procedure as a training device 
will afford the child the use of rioher language patterns and in 
some measure assist him in the organization of his thoughts* The 
doze procedure also offers sufficient structure so that the individual 
makes a miniscule number of syntactical errors* In the present study 
the number of syntactical errors made In the cloze procedure materials 
is less than one percent. However, one would expeot that syntactical 
* errors would increase as the fading inoreases. Since eaoh student's 

paper would be graded for syntax as well as for other variables, these 
errors would be detected early and the child oould then profit by 
syntactic doze training. 

Compreh ensi on and th^ Cn.o7.oPpo^ft<jirpR^ The present study 
with the cloze procedure indicated no growth in reading comprehension 
on the part of the participants in the experiment. Other experiments 
cited before (ffloomer 1 ) which handled the cloze procedure in a different 
manner found that doze procedure appeared, at least at the college and 
the sixth grade levels, to be related to growth in reading comprehension. 
A aeoent study by Sohneyer 34 , however, found no particular effects from 
prolonged doze procedure. The difference in these studies indicate that 
the manner in which the doze procedure is presented will either produce 
or not produoe effective results in increasing reading comprehension. 
Carefully prepared motivational, conditions are required for the production 
of reading comprehension. The question as to whether results in increased 
reading comprehension under motivational conditions are a function of 
doze procedure or merely a function of the motivational conditions as 
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applied to any type of reading comprehension material remains to be 
answered* 

The motivational oomponents which appeared to be effective when 
used in conjunction with the dose procedure are the following* 

The first is the use of reinforced dose procedure* This involves 
the use of a oorreot answer list used by the individual to score his 
material subsequent to completion of the doze procedure exercise* The 
answer list gives the word which the author put into the particular 
place in the paragraph and henoe the student receives an immediate score 
by grading the paper himself, and furthermore obtains specific feedback 
in relation to the various blanks which he had questions about* 

The second motivational factor is regrading for meaning by the 
instructor* This can be accomplished by having the instructor collect 
the papers at the termination of the grading by the student, and in turn, 
using a somewhat broader key which has not only the correct word written 
by the spedfio author but also other words of appropriate meaning. The 
instructor’s grade is typically somewhat higher than that of the student 
himself, hence giving additional reinforcement. when returned to the 
student* In this sense, the student first receives reinforcement for 
completion of the material in a manner meaningful to him; secondly, 
reinforcement for his approximation of the author’s language; and third, 
reinforoement from the instructor when the papers graded for oorreot 
meaning are returned to him* It also appears that some form of chart 
to make comparative scores available to the student so that he can watch 
his progress through the doze procedure material is of value. He can 
thus reinforce himself as he progresses in terms of the number of oorreot 




responses. 
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Third, the material should be graded In difficulty; hence the 
Individual will start with relatively simple material and continue 
until he is reading and filling in material which is very dose to his 
present reading level. The Individual can therefore watch himself 
progressing In the grade level of his ability as he continues through 
the program. 

Fourth, movement up the grade level should be contingent upon 
the number of correct responses made by the individual student* Basic* 
ally then the material should be supplied to the student In such a 
manner that there are several bits of material in depth at any partic- 
ular grade level* The criterion for movement should be rdatively high, 
approaching 90 to 95$ correct responses. The individual who falls to 
achieve such a score will therefore remfiin with material at the same 
grade level. The Individual who achieves at the 90 or 95$ correct 
response levd will move to a levd of more difficult material* Thus 
the individual student’s rate through the program is contingent upon 
the quality of his responses* The Individual who does very well is 
reinforced by a number of correct responses, and is also reinforced by 
nls movement from grade level to grade lev«>l through the end of the 
program. 

At this point the content of the material in terms of its Interest 
to the individual appears to be irrelevant. More work would need to be 
done to establish whether or not the effectiveness of the material can 
be controlled by the Interest level of the material. 

There appears to be a further limitation in relation to the age 
group for which this material is effective* Close procedure will not work 
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for children who cannot spell or write in th3 appropriate missing words, 
though an attempt at the use of oral dose with younger children might 
prove to be interesting. Farther limitation is that dose procedure 
is ineffective with pupils who have not devdoped fairly strong language 
patterns. Younger children appear to repeat the doze cue rather than 
* word which will make the material meaningful and hence do 
not develop systematic meaningfUlness from dose material. A mental 
age of about ten or higher is required to function adequately with 
dose procedure material. Below this level the probability of making 
errors because of poorly devdoped ability to manipulate language is 
extremely high, resulting in frustration and producing no real benefits 
in aoility to read. Therefore above a certain levd children will learn 
to comprehend more effectually with properly designed dose materials. 
Here again it must be cautioned that more experimentation needs to be 
done with the doze procedure as a reading comprehension device. 



CHose Pr ocedure and 




Experiments with dose 



procedure indicate that doze procedure is effective for increasing the 
immediate comprehension of materials. The present study indicates only 



mild relationships occur between comprehension increases and dose pro 



oedure with the use of non-reinforced dose procedure. The use of dose 
procedure to increase understanding of content should be reinforced. 



Pilot studies show that tetremely difficult doze procedure material is 
ineffective in teaching oontent. 



If we look at dose procedure in rdation to programed instruc- 
tion, we find that reinforced dose procedure has all the dements of 
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good programed instruction. First, the close prooedure involves 
content material and oan be used to deliver oontent to an individual, 
a prime requisite of programed instruction. 

Seoond, doze prooedure demands responses of individuals, vhioh 
the seoond oriterlon for programed instruction { and third, doze 
prooedure presents reinforcement for correct responses. 

Some limitations in relation to the organization of the material 

made into doze oontent material should be expressed, lhe material 

must move oarefully and sequentially from lower order concepts to 

higher order oonoepts. It is often difficult to find written material 
of this nature., 

The seoond oriterlon would be that the material be relatively 

redUndant produoa * *** order of correct responses on the part of 
the Individ-*.. inis again tends to li m it the amount of material 

available. However, materld certainly could be easily written to meet 
these criteria* 

Third, doze deletions of teaching material oontent should 
involve the use of nouns, verbs and adjectives in greater proportion 
than the fhnetion words, beoause we are concerned with the individual's 
ability to discover and fathom the carriers of meaning wither than their 
syntactic relationship in particular situations. Occasionally, however, 
a significant word might fall in one of the function word categories. 

It should be noted that the selection of dosure, is not random but 
rather ddeUons are made on the basis, first, of their importance as 

<*«ler« of meaning, and second, their presence in other situations or 
their value as solutions to conceptual problems. 
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The oloze procedure will probably be somewhat less effective 
than programed instruction of the Skinnerian type in producing verbal 
responses patterned directly upon the content of the material. On the 
other hand, dose procedure can be manipulated such that the individual 
will be more likely to conceptualize the material. 

For the purpose of increasing the probability of reinforcement 
a key word list (a scrambled list of all the oorrect answers) may be 
placed upon the page so the individual learner can use the list as an 
aid to achieving correct responses. Reinforcement with a key word list 
would be the specific word deleted, particularly where noun deletions 
are used. With deletions of adjectives and verbs, the instructor may 
use a second scoring key which would allow adequate substitutes. 

As with other applications, the use of doze procedure as a 
technique for increasing an individual's knowledge of content would 
require considerably more experimentation . Certain indications in the 
literature appear to suggest that doze procedure will increase immedi- 
ate comprehension, bet there must also be a concern with the delayed 
recall of content material learned in a doze procedure exercise. This 
appears to be an aree of doze procedure which is worthy of experimental 
effort, 

SUMMARY 

A large number of potential applications of doze procedure, 
eaob requiring considerably more researoh, are evident. 

First, standardisation of doze procedure as a readability 
measurement instrument and as a reading comprehension device would 
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make it a more useful tool for the classroom teacher* 

Second* dose can be adapted to a number of other testing 
situations— as an IQ test; a specific aptitude test for career 
advisement; a test of standard syntactic skill? and as a projective 
devioe* appear to have great promise* 

Third* dose as a teaching instrument appears to have 
potential for teaching standard syntax* composition* reading compre- 
hension and content* 

In each case exploration of an avenue of dose application 
should be conducted in strict experimental form* 
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